Happiness detection with facial physiological measurement from hyperspectral imaging. (October 2022)

• Record Type: Journal Article
• Authors: Hao, Min; Zhang, Huiling; Liu, Guangyuan; Gokhale, Anu; Xu, Ya; Cai, Jing; Xie, Desheng
• Is Part Of: Computers & electrical engineering. Volume 103(2022)
• Publication Date: 2022-10
• Subjects: Facial analysis -- Happiness detection -- Hyperspectral imaging -- Spectral Reflectance -- Tissue oxygen saturation; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.108332
• Languages: English
• ISSNs: 0045-7906

Abstract

Highlights: The variations of StO2 are not necessarily consistent even in some symmetric facial regions when subject's emotion changes. The signature intensities in anger are usually higher than in happiness in most ROIs with some significant differences. Substantial experimental results demonstrate that using StO2 extracted from partial facial ROIs to detect human emotion is technically feasible. Different ROIs manifest different contributive discrimination performance on emotion recognition. Abstract: Hyperspectral imaging technology (HSI) can be used to remotely detect emotion levels in humans based on the tissue oxygen saturation (StO2 ) from their faces. Our prior research demonstrated some significant differences among the variations of StO2 in different facial regions of interest (ROIs) when the subjects are truly happy. Experiment is conducted for data collection with HSI where the subjects (a total of fifty participants, who rigidly conform to the research protocol) are required to show spontaneous emotions (calm, happy, and unhappy: angry). Based on the comprehensive analysis of musculoskeletal anatomy and facial action coding system (FACS), 19 ROIs have been determined. Consequently, a 19 dimensional StO2 vector is constructed as a feature input instead of employing the entire facial region. Lastly, experimental results through basic K -Nearest Neighbor (KNN) report an average accuracy rate of 83.32% when only considering fewer StO2 features. In addition, some experiments using other advanced machine learning methods also validate its effective recognition performance with facial StO2 as an affective indicator. Our findings demonstrate the possible practical feasibility of StO2 in happiness detection. Graphical abstract: Image, graphical abstract