Driving behavior explanation with multi-level fusion. (March 2022)

• Record Type: Journal Article
• Authors: Ben-Younes, Hédi; Zablocki, Éloi; Pérez, Patrick; Cord, Matthieu
• Is Part Of: Pattern recognition. Volume 123(2022)
• Publication Date: 2022-03
• Subjects: Explainable self-driving -- Multi-level fusion -- Cause classification -- Natural language explanations -- HDD -- BDD-X; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108421
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A deep architecture, called BEEF, explains the behavior of a trajectory prediction model, in an online fashion. The core module discovers fine-grained correlation between high-level decision features and intermediate perceptual features, with a BLOCK fusion module. The global approach is flexible with respect to various settings (online/offline explanations, cause classification / natural language justification). Abstract: In this era of active development of autonomous vehicles, it becomes crucial to provide driving systems with the capacity to explain their decisions. In this work, we focus on generating high-level driving explanations as the vehicle drives. We present BEEF, for BEhavior Explanation with Fusion, a deep architecture which explains the behavior of a trajectory prediction model. Supervised by annotations of human driving decisions justifications, BEEF learns to fuse features from multiple levels. Leveraging recent advances in the multi-modal fusion literature, BEEF is carefully designed to model the correlations between high-level decisions features and mid-level perceptual features. The flexibility and efficiency of our approach are validated with extensive experiments on the HDD and BDD-X datasets.