Car crash detection using ensemble deep learning and multimodal data from dashboard cameras. (30th November 2021)

• Record Type: Journal Article
• Authors: Choi, Jae Gyeong; Kong, Chan Woo; Kim, Gyeongho; Lim, Sunghoon
• Is Part Of: Expert systems with applications. Volume 183(2021)
• Publication Date: 2021-11-30
• Subjects: Dashboard camera -- Car crash -- Multimodal data -- Deep learning -- Ensemble technique; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115400
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A car crash detection system based on an ensemble deep learning model is proposed. An ensemble deep learning model uses video and audio data from dashboard cameras. A GRU-and-CNN-based base classifier is developed for video data. A GRU-based and a CNN-based base classifiers are developed for audio data. The proposed system establishes state-of-the-art classification performances. Abstract: Due to the increase in motor vehicle accidents, there is a growing need for high-performance car crash detection systems. The authors of this research propose a car crash detection system that uses both video data and audio data from dashboard cameras in order to improve car crash detection performance. While most existing car crash detection systems depend on single modal data (i.e., video data or audio data only), the proposed car crash detection system uses an ensemble deep learning model based on multimodal data (i.e., both video and audio data), because different types of data extracted from one information source (e.g., dashboard cameras) can be regarded as different views of the same source. These different views complement one another and improve detection performance, because one view may have information that the other view does not contain. In this research, deep learning techniques, gated recurrent unit (GRU) and convolutional neural network (CNN), are used to develop a car crash detection system. A weighted average ensemble is used as an ensemble technique. The proposed car crash detection system, which is based on multiple classifiers that use both video and audio data from dashboard cameras, is validated using a comparison with single classifiers that use video data or audio data only. Car accident YouTube clips are used to validate this research. The experimental results indicate that the proposed car crash detection system performs significantly better than single classifiers. It is expected that the proposed car crash detection system can be used as part of an emergency road call service that recognizes traffic accidents automatically and allows immediate rescue after transmission to emergency recovery agencies.