An end‐to‐end framework for the detection of mathematical expressions in scientific document images. Issue 1 (6th September 2021)

• Record Type: Journal Article
• Authors: Phong, Bui Hai; Hoang, Thang Manh; Le, Thi‐Lan
• Is Part Of: Expert systems. Volume 39:Issue 1(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-09-06
• Subjects: deep neural network -- distance transform -- Faster R‐CNN -- machine learning -- mathematical expression detection; Expert systems (Computer science); 006.33
• DOI: 10.1111/exsy.12800
• Languages: English
• ISSNs: 0266-4720

Abstract

Abstract: The detection of mathematical expressions is a prerequisite step for the digitisation of scientific documents. Many different multistage approaches have been proposed for the detection of expressions in document images, that is, page segmentation and expression detection. However, the detection accuracy of such methods still needs improvement owing to errors in the page segmentation of complex documents. This paper presents an end‐to‐end framework for mathematical expression detection in scientific document images without requiring optical character recognition (OCR) or document analysis techniques applied in conventional methods. The novelty of this paper is twofold. First, because document images are usually in binary form, the direct use of these images, which lack texture information as input for detection networks, may lead to an incorrect detection. Therefore, we propose the application of a distance transform to obtain a discriminating and meaningful representation of mathematical expressions in document images. Second, the transformed images are fed into the faster region with a convolutional neural network (Faster R‐CNN) optimized to improve the accuracy of the detection. The proposed framework was tested on two benchmark data sets (Marmot and GTDB). Compared with the original Faster R‐CNN, the proposed network improves the accuracies of detection of isolated and inline expressions by 5.09% and 3.40%, respectfully, on the Marmot data set, whereas those on the GTDB data set are improved by 4.04% and 4.55%. A performance comparison with conventional methods shows the effectiveness of the proposed method.