Statistical learning for OCR error correction. Issue 6 (November 2018)

• Record Type: Journal Article
• Authors: Mei, Jie; Islam, Aminul; Moh'd, Abidalrahman; Wu, Yajing; Milios, Evangelos
• Is Part Of: Information processing & management. Volume 54:Issue 6(2018:Nov.)
• Page Start: 874
• Page End: 887
• Publication Date: 2018-11
• Subjects: OCR post-processing -- OCR error -- Error correction -- Statistical learning; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2018.06.001
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Modern OCR engines incorporate some form of error correction, typically based on dictionaries. However, there are still residual errors that decrease performance of natural language processing algorithms applied to OCR text. In this paper, we present a statistical learning model for post-processing OCR errors, either in a fully automatic manner or followed by minimal user interaction to further reduce error rate. Our model employs web-scale corpora and integrates a rich set of linguistic features. Through an interdependent learning pipeline, our model produces and continuously refines the error detection and suggestion of candidate corrections. Evaluated on a historical biology book with complex error patterns, our model outperforms various baseline methods in the automatic mode and shows an even greater advantage when involving minimal user interaction. Quantitative analysis of each computational step further suggests that our proposed model is well-suited for handling volatile and complex OCR error patterns, which are beyond the capabilities of error correction incorporated in OCR engines.