Accurate recognition of words in scenes without character segmentation using recurrent neural network. (March 2017)

• Record Type: Journal Article
• Authors: Su, Bolan; Lu, Shijian
• Is Part Of: Pattern recognition. Volume 63(2017:Mar.)
• Page Start: 397
• Page End: 405
• Publication Date: 2017-03
• Subjects: Scene text recognition -- Recurrent neural network; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.10.016
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: Recognition of texts in scenes is one of the most important tasks in many computer vision applications. Though different scene text recognition techniques have been developed, scene text recognition under a generic condition is still a very open and challenging research problem. One major factor that defers the advance in this research area is character touching, where many characters in scene images are heavily touched with each other and cannot be segmented for recognition. In this paper, we proposed a novel scene text recognition technique that performs word level recognition without character segmentation. Our proposed technique has three advantages. First it converts each word image into a sequential signal for the scene text recognition. Second, it adapts the recurrent neural network (RNN) with Long Short Term Memory (LSTM), the technique that has been widely used for handwriting recognition in recent years. Third, by integrating multiple RNNs, an accurate recognition system is developed which is capable of recognizing scene texts including those heavily touched ones without character segmentation. Extensive experiments have been conducted over a number of datasets including several ICDAR Robust Reading datasets and Google Street View dataset. Experiments show that the proposed technique is capable of recognizing texts in scenes accurately. Abstract : Highlights: We designed a novel method that converts a word image into a sequential signal. We designed an ensembling RNN for word-level scene text recognition which obtained superior recognition accuracy. Our method uses publicly available datasets in training which provides a baseline for benchmarking of the future works. Our method uses word instead character level annotations, which reduces the efforts in ground truth generation greatly.