Intelligent typo correction for text mining through machine learning. Issue 2 (2015)

• Record Type: Journal Article
• Authors: Huang, Yinghao; Murphey, Yi Lu; Ge, Yao
• Is Part Of: International journal of knowledge engineering and data mining. Volume 3:Issue 2(2015)
• Page Start: 115
• Page End: 142
• Publication Date: 2015
• Subjects: machine learning -- neural networks -- spelling errors -- text processing -- text mining -- intelligent correction -- typo correction -- typos -- typographical errors -- typo detection -- grammatical errors -- text documents -- self-invented terminology -- spell checking -- language knowledge -- domain-specific knowledge -- machine learning -- case study -- automotive fault diagnosis -- automobile industry; Knowledge representation (Information theory) -- Periodicals; Data mining -- Periodicals; 006.305
• Languages: English
• ISSNs: 1755-2087

Abstract

Typo detection and correction is an important process in many text mining applications. This research focuses on automatic typo detection and correction for processing text documents that are unstructured, contain many grammar and spelling errors, and have many self-invented terminologies that can be interpreted only through domain-specific knowledge. In this paper we present an intelligent typo detection and correction (ITDC) system. Its 'intelligence' is reflected by automatically identifying and accurately correcting a broad range of typos, from simple typos such as duplication, omission, transposition, substitution characters, to complex spelling errors, such as word boundary errors, unconventional use of acronyms, etc. ITDC utilises general language knowledge and domain-specific knowledge extracted by machine learning algorithms. It is evaluated through a case study that involves the automatic processing of automotive fault diagnostic text documents. The experiment results show that the proposed system outperforms some of the state-of-art spell checking systems.