A text classification method based on LSTM and graph attention network. Issue 1 (31st December 2022)

• Record Type: Journal Article
• Authors: Wang, Haitao; Li, Fangbing
• Is Part Of: Connection science. Volume 34:Issue 1(2022)
• Page Start: 2466
• Page End: 2480
• Publication Date: 2022-12-31
• Subjects: Text classification -- LSTM -- graph attention network -- dependency syntax -- deep learning; Neural computers -- Periodicals; Artificial intelligence -- Periodicals; Cognitive science -- Periodicals; Connectionism -- Periodicals; 006.3
• DOI: 10.1080/09540091.2022.2128047
• Languages: English
• ISSNs: 0954-0091

Abstract

Abstract : Text classification is a popular research topic in the natural language processing. Recently solving text classification problems with graph neural network (GNN) has received increasing attention. However, current graph-based studies ignore the hidden information in text syntax and sequence structure, and it is difficult to use the model directly for processing new documents because the text graph is built based on the whole corpus including the test set. To address the above problems, we propose a text classification model based on long short-term memory network (LSTM) and graph attention network (GAT). The model builds a separate graph based on the syntactic structure of each document, generates word embeddings with contextual information using LSTM, then learns the inductive representation of words by GAT, and finally fuses all the nodes in the graph together into the document embedding. Experimental results on four datasets show that our model outperforms existing text classification methods with faster convergence and less memory consumption than other graph-based methods. In addition, our model shows a more notable improvement when using less training data. Our model proves the importance of text syntax and sequence information for classification results.