Quantifying the impact of dirty OCR on historical text analysis: Eighteenth Century Collections Online as a case study. (22nd April 2019)

• Record Type: Journal Article
• Authors: Hill, Mark J; Hengchen, Simon
• Is Part Of: Digital scholarship in the humanties. Volume 34:Number 4(2019)
• Page Start: 825
• Page End: 843
• Publication Date: 2019-04-22
• Subjects: Philology -- Data processing -- Periodicals; Computational linguistics -- Periodicals; 410.285
• DOI: 10.1093/llc/fqz024
• Languages: English
• ISSNs: 2055-768X

Abstract

Abstract: This article aims to quantify the impact optical character recognition (OCR) has on the quantitative analysis of historical documents. Using Eighteenth Century Collections Online as a case study, we first explore and explain the differences between the OCR corpus and its keyed-in counterpart, created by the Text Creation Partnership. We then conduct a series of specific analyses common to the digital humanities: topic modelling, authorship attribution, collocation analysis, and vector space modelling. The article concludes by offering some preliminary thoughts on how these conclusions can be applied to other datasets, by reflecting on the potential for predicting the quality of OCR where no ground-truth exists.