Driving behavior explanation with multi-level fusion. (March 2022) - Ben-Younes, Hédi; Zablocki, Éloi; Pérez, Patrick; Cord, Matthieu
Highlights: A deep architecture, called BEEF, explains the behavior of a trajectory prediction model, in an online fashion. The core module discovers fine-grained correlation between high-level decision features and intermediate perceptual features, with a BLOCK fusion module. The global approach is flexible with respect to various settings (online/offline explanations, cause classification / natural language justification). Abstract: In this era of active development of autonomous vehicles, it becomes crucial to provide driving systems with the capacity to explain their decisions. In this work, we focus on generating high-level driving explanations as the vehicle drives. We present BEEF, for BEhavior Explanation with Fusion, a deep architecture which explains the behavior of a trajectory prediction model. Supervised by annotations of human driving decisions justifications, BEEF learns to fuse features from multiple levels. Leveraging recent advances in the multi-modal fusion literature, BEEF is carefully designed to model the correlations between high-level decisions features and mid-level perceptual features. The flexibility and efficiency of our approach are validated with extensive experiments on the HDD and BDD-X datasets.