A learnable end-edge-cloud cooperative network for driving emotion sensing. (October 2022)

• Record Type: Journal Article
• Authors: Ding, Cheng; Ding, Fei; Gorbachev, Sergey; Yue, Dong; Zhang, Dengyin
• Is Part Of: Computers & electrical engineering. Volume 103(2022)
• Publication Date: 2022-10
• Subjects: End-edge-cloud human-machine collaboration -- Video stream key frames extraction -- Image reconstruction algorithm -- Region of interest -- Driver emotion recognition; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.108378
• Languages: English
• ISSNs: 0045-7906

Abstract

Highlights: Device-edge-cloud collaboration architecture. Keyframe extraction of video streams, remove non-keyframes. Graphics compression and reconstruction algorithms. Extract the region of interest of the face. Abstract: The driver's emotional state directly affects safe driving. Under the "vehicle-human-road-cloud" integrated control framework, we propose an end-edge-cloud collaborative emotion perception network model (EEC-Net). The end side extracts the key frames of the driver's face video stream and performs batch compression; the edge side extracts the region of interest (ROI) of the reconstructed images as the input of the emotion recognition model (tiny_Xception) for classification; the cloud control terminal receives abnormal ROI image data and performs online training to dynamically adjust the operating parameters of the edge model. Finally, we test on open and self-built datasets, and the results show tiny_Xception has a significant improvement in accuracy of 2.45% compared to mini_Xception; the EEC-Net model can reliably perceive the negative emotion period, and the overall system memory consumption is reduced by about 5%, and the network transmission data volume and the computation time of emotion recognition are reduced by 95%, 60% respectively. Graphical abstract: Image, graphical abstract