Driving maneuver early detection via sequence learning from vehicle signals and video images. (July 2020)

• Record Type: Journal Article
• Authors: Peng, Xishuai; Murphey, Yi Lu; Liu, Ruirui; Li, Yuanxiang
• Is Part Of: Pattern recognition. Volume 103(2020:Jul.)
• Publication Date: 2020-07
• Subjects: Driving maneuver early detection -- Deep neural networks -- Sequence learning -- Advanced driver assistance systems -- Computer vision; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107276
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel deep sequential model UMD-DMED is proposed for early detection of five driving maneuver classes: left turn, right turn, left lane change, right lane change, and driving straight. The proposed UMD-DMED model contains three major innovative computational components, distance-based representation of driving context, combined vehicle trajectory features and scene-centric features, and a focal loss based LSTM model. The extensive experiments are conducted using a data set containing 1078 maneuver events extracted from 37 hours of real-world driving trips from 7 different drivers. The proposed UMD-DMED achieved better detection performances and earlier detection time for detecting four classes of driving maneuvers, left and right turns, and left and right lane changes, than other advanced driving maneuver detection algorithms. Abstract: Driving Maneuver Early Detection (DMED) is particularly useful for many applications of intelligent vehicle systems, including driver warning and collision avoidance systems. In this paper, we introduce a robust DMED model, denoted as University of Michigan Dearborn (UMD)-DMED, developed using innovative features and deep learning techniques. The UMD-DMED model contains three major computational components, distance based representation of driving context, combined vehicle trajectory features and visual features, and a Long Short-Term Memory (LSTM)-based neural network that captures temporal dependencies of driving maneuvers. To properly evaluate the performances of UMD-DMED, we developed two DMED systems based on the UMD-DMED model, one system is based on partially observed evidence of maneuver events, and another on features observed ahead of the time that driving maneuvers take place. We conducted the extensive experiments using a data set containing 1078 maneuver events extracted from 37 hours of real world driving trips. The results demonstrate that the UMD-DMED model is capable of learning the latent features of five different classes of driving maneuvers, i.e. left turn, right turn, left lane change, right lane change, driving straight . Comparing to four different state-of-the-art DMED systems, the UMD-DMED achieved better detection performances in both, the detection based on partial observations of driver maneuvering, and based on driving context observed ahead-of-time.