A two-stage deep neural network for multi-norm license plate detection and recognition. (1st December 2019)

• Record Type: Journal Article
• Authors: Kessentini, Yousri; Besbes, Mohamed Dhia; Ammar, Sourour; Chabbouh, Achraf
• Is Part Of: Expert systems with applications. Volume 136(2019)
• Page Start: 159
• Page End: 170
• Publication Date: 2019-12-01
• Subjects: License plate detection and recognition -- Semi-automatic annotation -- Convolutional neural networks -- Recurrent neural networks -- YOLO -- Deep learning; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2019.06.036
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A two-stage multi-norm LP detection and recognition system is proposed. A semi-automatic annotation procedure of LP images is proposed. A new public multi-norm LP images dataset is provided freely for research purpose. The proposed system is robust and efficient in real-time applications. Graphical abstract: Abstract: In this work, we tackle the problem of multi-norm and multilingual license plate (LP) detection and recognition in natural scene images. The system architecture use a pipeline with two deep learning stages. The first network was trained to detect license plates on the full raw image by using the latest state-of-the-art deep learning based detector namely YOLOv2. The second stage is then applied on the cropped image to recognize captured license plate photographs. Two recognition engines are compared in this work: a segmentation-free approach based on a convolutional recurrent neural network where the recognition is carried out over the entire LP image without any prior segmentation and a joint detection/recognition approach that performs the recognition on the plate component level. We also introduced a new large-scale dataset for automatic LP recognition that includes 9.175 fully annotated images. In order to reduce the time and cost of annotation processing, we propose a new semi-automatic annotation procedure of LP images with labeled components bounding box. The proposed system is evaluated using two datasets collected from real road surveillance and parking access control environments. We show that the system using two YOLO stages performs better in the context of multi-norm and multilingual license plate. Additional experiments are conducted on the public AOLP dataset and show that the proposed approach outperforms over other existing state-of-the-art methods.