Batch-adaptive rejection threshold estimation with application to OCR post-processing. Issue 21 (30th November 2015)

• Record Type: Journal Article
• Authors: Navarro-Cerdan, J. Ramon; Arlandis, Joaquim; Llobet, Rafael; Perez-Cortes, Juan-Carlos
• Is Part Of: Expert systems with applications. Volume 42:Issue 21(2015)
• Page Start: 8111
• Page End: 8122
• Publication Date: 2015-11-30
• Subjects: Rejection threshold -- OCR post-processing -- Language models -- Weighted finite-state transducers -- Error vs. cost curve -- Cumulative error vs. cost curve -- OCR error-generation model; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2015.06.022
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: OCR strings are post-processed to comply with language constraints. A user-defined target error rate is met on a batch of OCR post-processed strings. The rejection threshold is automatically estimated from the target error rate. The threshold can be estimated for language models using real or synthetic samples. Accurate error rate estimation on test samples having different confidence distributions. Abstract: An OCR process is often followed by the application of a language model to find the best transformation of an OCR hypothesis into a string compatible with the constraints of the document, field or item under consideration. The cost of this transformation can be taken as a confidence value and compared to a threshold to decide if a string is accepted as correct or rejected in order to satisfy the need for bounding the error rate of the system. Widespread tools like ROC, precision-recall, or error-reject curves, are commonly used along with fixed thresholding in order to achieve that goal. However, those methodologies fail when a test sample has a confidence distribution that differs from the one of the sample used to train the system, which is a very frequent case in post-processed OCR strings (e.g., string batches showing particularly careful handwriting styles in contrast to free styles). In this paper, we propose an adaptive method for the automatic estimation of the rejection threshold that overcomes this drawback, allowing the operator to define an expected error rate within the set of accepted (non-rejected) strings of a complete batch of documents (as opposed to trying to establish or control the probability of error of a single string ), regardless of its confidence distribution. The operator (expert) is assumed to know the error rate that can be acceptable to the user of the resulting data. The proposed system transforms that knowledge into a suitable rejection threshold. The approach is based on the estimation of an expected error vs. transformation cost distribution. First, a model predicting the probability of a cost to arise from an erroneously transcribed string is computed from a sample of supervised OCR hypotheses. Then, given a test sample, a cumulative error vs. cost curve is computed and used to automatically set the appropriate threshold that meets the user-defined error rate on the overall sample. The results of experiments on batches coming from different writing styles show very accurate error rate estimations where fixed thresholding clearly fails. An original procedure to generate distorted strings from a given language is also proposed and tested, which allows the use of the presented method in tasks where no real supervised OCR hypotheses are available to train the system.