Recognition of visual-related non-driving activities using a dual-camera monitoring system. (August 2021)

• Record Type: Journal Article
• Authors: Yang, Lichao; Dong, Kuo; Ding, Yan; Brighton, James; Zhan, Zhenfei; Zhao, Yifan
• Is Part Of: Pattern recognition. Volume 116(2021)
• Publication Date: 2021-08
• Subjects: Driver behaviour -- Level 3 automation -- Computer vision -- Non-driving related task -- activities identification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107955
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Eye gaze mapping using a cost-effective dual-camera system. Monitoring of more diverse and uncertain driver behaviours. A sliding time window technique for activity classifer. Recognition of visual attention-related NDAs for driving safety enhancement. Abstract: For a Level 3 automated vehicle, according to the SAE International Automation Levels definition (J3016), the identification of non-driving activities (NDAs) that the driver is engaging with is of great importance in the design of an intelligent take-over interface. Much of the existing literature focuses on the driver take-over strategy with associated Human-Machine Interaction design. This paper proposes a dual-camera based framework to identify and track NDAs that require visual attention. This is achieved by mapping the driver's gaze using a nonlinear system identification approach, on the object scene, recognised by a deep learning algorithm. A novel gaze-based region of interest (ROI) selection module is introduced and contributes about a 30% improvement in average success rate and about a 60% reduction in average processing time compared to the results without this module. This framework has been successfully demonstrated to identify five types of NDA required visual attention with an average success rate of 86.18%. The outcome of this research could be applicable to the identification of other NDAs and the tracking of NDAs within a certain time window could potentially be used to evaluate the driver's attention level for both automated and human-driving vehicles.