Text detection and script identification in natural scene images using deep learning. (May 2021)

• Record Type: Journal Article
• Authors: Khalil, Ashwaq; Jarrah, Moath; Al-Ayyoub, Mahmoud; Jararweh, Yaser
• Is Part Of: Computers & electrical engineering. Volume 91(2021)
• Publication Date: 2021-05
• Subjects: Text detection -- Script identification -- Natural scene images -- Deep learning -- Fully convolution network -- Undersampling -- Oversampling; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2021.107043
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: The detection of text in an image and identification of its language are important tasks in optical character recognition. Such tasks are challenging, particularly in natural scene images. Previous studies have been conducted with a focus on convolutional neural networks for script identification. In other studies, fully convolutional networks (FCNs) have been used for model enhancement and not as classifiers. In this study, we use FCNs for both model enhancement and classification. The proposed methodology improves the Efficient and Accurate Scene Text Detector by adding new FCN branches for script identification. Moreover, whereas most end-to-end (e2e) methods train the text detection and script identification models separately, we propose two e2e methods for jointly training the models, namely, multi-channel mask (MCM) and multi-channel segmentation (MCS). The results show that the performance of an MCM is similar to that of other state-of-the-art methods, whereas MCS outperforms existing methods with recall values of 54.34% and 81.13%, when using the ICDAR MLT 2017 and MLe2e datasets, respectively. Graphical abstract: Highlights: Detecting and identifying scripts in natural scene images are challenging. A new framework is developed for multi-lingual text detection and identification. The framework is developed using the deep convolutional neural networks. Different natural scene image datasets are used for evaluating the framework.