Automatic large-scale data acquisition via crowdsourcing for crosswalk classification: A deep learning approach. (November 2017)

• Record Type: Journal Article
• Authors: Berriel, Rodrigo F.; Rossi, Franco Schmidt; de Souza, Alberto F.; Oliveira-Santos, Thiago
• Is Part Of: Computers & graphics. Volume 68(2017)
• Page Start: 32
• Page End: 42
• Publication Date: 2017-11
• Subjects: Crosswalk classification -- Zebra-crossing classification -- Deep learning -- Crowdsourcing -- Large-scale database; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2017.08.004
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: Exploitation of crowdsourcing platforms, such as OpenStreetMap and Google StreetView. Automatic acquisition and annotation of a large-scale database (+500, 000 images). Deep learning (ConvNet) applied on the crosswalk classification problem. Cross-database evaluation indicates the system is ready for real-world applications. Graphical abstract: The proposed system exploits crowdsourcing platforms to automatically acquire and annotate a large-scale dataset, and train a Convolutional Neural Network to perform crosswalk classification. Abstract: Correctly identifying crosswalks is an essential task for the driving activity and mobility autonomy. Many crosswalk classification, detection and localization systems have been proposed in the literature over the years. These systems use different perspectives to tackle the crosswalk classification problem: satellite imagery, cockpit view (from the top of a car or behind the windshield), and pedestrian perspective. Most of the works in the literature are designed and evaluated using small and local datasets, i.e. datasets that present low diversity. Scaling to large datasets imposes a challenge for the annotation procedure. Moreover, there is still need for cross-database experiments in the literature because it is usually hard to collect the data in the same place and conditions of the final application. In this paper, we present a crosswalk classification system based on deep learning. For that, crowdsourcing platforms, such as OpenStreetMap and Google Street View, are exploited to enable automatic training via automatic acquisition and annotation of a large-scale database. Additionally, this work proposes a comparison study of models trained using fully-automatic data acquisition and annotation against models that were partially annotated. Cross-database experiments were also included in the experimentation to show that the proposed methods enable use with real world applications. Our results show that the model trained on the fully-automatic database achieved high overall accuracy (94.12%), and that a statistically significant improvement (to 96.30%) can be achieved by manually annotating a specific part of the database. Finally, the results of the cross-database experiments show that both models are robust to the many variations of image and scenarios, presenting a consistent behavior.