Character convolutions for Arabic Named Entity Recognition with Long Short-Term Memory Networks. (November 2019)

• Record Type: Journal Article
• Authors: Khalifa, Muhammad; Shaalan, Khaled
• Is Part Of: Computer speech & language. Volume 58(2019)
• Page Start: 335
• Page End: 346
• Publication Date: 2019-11
• Subjects: Named Entity Recognition -- Arabic -- Recurrent Neural Network -- LSTM -- Convolutional Neural Network; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2019.05.003
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Examine an LSTM neural tagging model for Named Entity Recognition (NER). Demonstrate the capability of the proposed model with both word and character levels and how it can handle out-of-vocabulary words. Convolutional Neural Network can be one of the promising technologies that can successfully extract character-level representations. Compare different word vector models for the task of Arabic NER. Conduct experimental evaluation that shows that our model performance outperforms state-of-the-art Arabic NER systems on many standard benchmarks. Abstract: Named Entity Recognition (NER) is a significant information extraction task since it is an important component of many natural language processing applications, such as Information Retrieval, Question Answering and Speech Recognition. The complexity and morphological richness of the Arabic language is the main reason why most existing Arabic NER systems rely strongly on hand-crafted feature engineering. In this paper, we propose to augment the existing LSTM neural tagging model for Arabic NER with a Convolutional Neural Network (CNN) for the extraction of relevant character-level features. By operating on the character-level, the proposed model is able to handle out-of-vocabulary words. Our results show that character CNN is able to outperform the previously used character-level Bi-directional Long Short-Term Memory Networks (BiLSTM) in many settings. Moreover, our observations indicate that CNNs tend to perform better than BiLSTM on relatively longer tokens. In addition, we conduct a comparison of four different pre-trained word vector models for Arabic NER and results show that a Skip-Gram Word2vec model, pre-trained on a subset of the Arabic Gigaword corpus, is generally sufficient to obtain acceptable Arabic NER performance.