Deep traffic light detection by overlaying synthetic context on arbitrary natural images. (February 2021)

• Record Type: Journal Article
• Authors: Vieira de Mello, Jean Pablo; Tabelini, Lucas; F. Berriel, Rodrigo; M. Paixão, Thiago; F. de Souza, Alberto; Badue, Claudine; Sebe, Nicu; Oliveira-Santos, Thiago
• Is Part Of: Computers & graphics. Volume 94(2021)
• Page Start: 76
• Page End: 86
• Publication Date: 2021-02
• Subjects: Traffic light -- Synthetic data -- Deep learning -- Object detection -- Image context; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2020.09.012
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: Use non-realistic computer graphics to generate training samples for object detection. Investigate the impact of context when training deep models with synthetic samples. Experiments are performed in several well-known traffic light datasets. Our approach achieves results comparable to those that use real-world training data. Graphical abstract: Abstract: Deep neural networks come as an effective solution to many problems associated with autonomous driving. By providing real image samples with traffic context to the network, the model learns to detect and classify elements of interest, such as pedestrians, traffic signs, and traffic lights. However, acquiring and annotating real data can be extremely costly in terms of time and effort. In this context, we propose a method to generate artificial traffic-related training data for deep traffic light detectors. This data is generated using basic non-realistic computer graphics to blend fake traffic scenes on top of arbitrary image backgrounds that are not related to the traffic domain. Thus, a large amount of training data can be generated without annotation efforts. Furthermore, it also tackles the intrinsic data imbalance problem in traffic light datasets, caused mainly by the low amount of samples of the yellow state. Experiments show that it is possible to achieve results comparable to those obtained with real training data from the problem domain, yielding an average mAP and an average F1-score which are each nearly 4 p.p. higher than the respective metrics obtained with a real-world reference model.