Dependency syntax guided BERT-BiLSTM-GAM-CRF for Chinese NER. (15th June 2022)

• Record Type: Journal Article
• Authors: Li, Daiyi; Yan, Li; Yang, Jianzhong; Ma, Zongmin
• Is Part Of: Expert systems with applications. Volume 196(2022)
• Publication Date: 2022-06-15
• Subjects: Name entity recognition -- Deep learning -- Dependency syntactic parsing -- BiLSTM -- Attention mechanism; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.116682
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Proposing a deep learning model to complete Chinese NER task. Exploiting a new method to capture the long-distance dependency between words. Combining semantic and syntactic information of words to solve polysemy problem. Demonstrating the model's excellent performance in Chinese NER task. Abstract: Traditional named entity recognition (NER) methods derived from deep learning (DL) often ignore the long-distance syntactic dependence relationships between words. In such methods, the representation of Chinese word vectors is too simple to solve the problem of polysemy. To address these shortcomings, this paper proposes a syntactic dependency guided BERT-BiLSTM-GAM-CRF model for Chinese NER. First, the self-attention method guided by the dependency syntactic parsing tree is embedded in the transformer computing framework of the BERT model. This can not only obtain the deep two-way linguistic representation of a word according to the context information of the word, but it can also better express the long-distance syntactic dependency relationships between words; Second, the trained word vector sequence is input into the BiLSTM layer embedded in the global attention mechanism (GAM), and then the most important whole situation semantic information in the sentence is obtained. Finally, the CRF is employed to learn the dependence relationships between adjacent labels to obtain the best sentence level label sequence. A large number of experiments on the CLUENER-2020 corpus, MSRA corpus, Weibo corpus and OntoNotes4 corpus prove that the constructed model has good results for the Chinese NER task, and the F1 values are 81.08%, 94.97%, 63.60% and 75.64% respectively.