A quadrilateral scene text detector with two-stage network architecture. (June 2020)

• Record Type: Journal Article
• Authors: Wang, Siwei; Liu, Yudong; He, Zheqi; Wang, Yongtao; Tang, Zhi
• Is Part Of: Pattern recognition. Volume 102(2020:Jun.)
• Publication Date: 2020-06
• Subjects: Scene text detection -- Deep learning -- Quadrilateral regression; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107230
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a novel quadrilateral regression algorithm for generating quadrilateral proposals and text detections. We introduce a dual-branch structure of a quadrilateral detection head and an additional rotated rectangle detection head to train the detection head at the second stage. We design a novel weighted RoI pooling module with learned weight masks to pool the feature. We propose an accelerated NMS algorithm for quadrilateral text proposals and detections. We integrate the above four novel modules into the two-stage architecture of Faster R-CNN, and achieve state-of-the-art results on multiple public datasets. Abstract: Many of the state-of-the-art methods can only localize scene texts with rotated rectangle boundaries, which may result in incorrect rectification of the detected scene texts and erroneous elimination of proposals or detections during non-maximum suppression (NMS). A few existing methods that can detect scene texts with quadrilateral boundaries, are just based on one-stage architectures or sliding windows scanning and thus have sub-optimal performance. To address these problems, we propose an end-to-end two-stage network architecture for scene text detection, which can accurately localize scene texts with quadrilateral boundaries. At the first stage, we propose a quadrilateral region proposal network (QRPN) for generating quadrilateral proposals, based on a newly proposed quadrilateral regression algorithm. At the second stage, we introduce a novel weighted RoI pooling module with learned weight masks to pool the features, and then classify the proposals and refine their shapes with the proposed quadrilateral regression algorithm again. Specially, during training, we adopt a dual-branch structure of detection heads, that is, jointly train the quadrilateral detection head and an additional rotated rectangle detection head. Furthermore, we develop an accelerated NMS algorithm with O( nlogn ) complexity, for redundant quadrilateral text proposals and detections eliminating during the first and the second stage, respectively. Experiments on several challenging benchmarks demonstrate the superior performance of the proposed method, which achieves state-of-the-art results on widely used benchmarks ICDAR 2017 MLT, RCTW, and ICDAR 2015 Incidental Scene Text benchmark.