Arabic spelling error detection and correction†. (18th March 2015)

• Record Type: Journal Article
• Authors: ATTIA, MOHAMMED; PECINA, PAVEL; SAMIH, YOUNES; SHAALAN, KHALED; VAN GENABITH, JOSEF
• Is Part Of: Natural language engineering. Volume 22:Part 5(2016)
• Page Start: 751
• Page End: 773
• Publication Date: 2015-03-18
• Subjects: Natural language processing (Computer science) -- Periodicals; Software engineering -- Periodicals; 006.35
• DOI: 10.1017/S1351324915000030
• Languages: English
• ISSNs: 1351-3249

Abstract

Abstract: A spelling error detection and correction application is typically based on three main components: a dictionary (or reference word list), an error model and a language model. While most of the attention in the literature has been directed to the language model, we show how improvements in any of the three components can lead to significant cumulative improvements in the overall performance of the system. We develop our dictionary of 9.2 million fully-inflected Arabic words (types) from a morphological transducer and a large corpus, validated and manually revised. We improve the error model by analyzing error types and creating an edit distance re-ranker. We also improve the language model by analyzing the level of noise in different data sources and selecting an optimal subset to train the system on. Testing and evaluation experiments show that our system significantly outperforms Microsoft Word 2013, OpenOffice Ayaspell 3.4 and Google Docs.