Hierarchical Self-Attention Hybrid Sparse Networks for Document Classification. (21st April 2021)

• Record Type: Journal Article
• Authors: Huang, Weichun; Tao, Ziqiang; Huang, Xiaohui; Xiong, Liyan; Yu, Jia
• Other Names: Liu Yu Academic Editor.
• Is Part Of: Mathematical problems in engineering. Volume 2021(2021)
• Publication Date: 2021-04-21
• Subjects: Engineering mathematics -- Periodicals; 510.2462
• DOI: 10.1155/2021/5594895
• Languages: English
• ISSNs: 1024-123X

Abstract

Abstract : Document classification is a fundamental problem in natural language processing. Deep learning has demonstrated great success in this task. However, most existing models do not involve the sentence structure as a text semantic feature in the architecture and pay less attention to the contexting importance of words and sentences. In this paper, we present a new model based on a sparse recurrent neural network and self-attention mechanism for document classification. Subsequently, we analyze three variant models of GRU and LSTM for evaluating the sparse model in different datasets. Extensive experiments demonstrate that our model obtains competitive performance and outperforms previous models.