Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. (October 2020)

• Record Type: Journal Article
• Authors: Patlar Akbulut, Fatma; Perros, Harry G.; Shahzad, Muhammad
• Is Part Of: Computer methods and programs in biomedicine. Volume 195(2020)
• Publication Date: 2020-10
• Subjects: Affect recognition -- Autoregressive hidden Markov models -- Machine learning -- Biosignals -- Heart rate variability; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2020.105571
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: A user-independent bimodal affect recognition method that uses AR-HMM for electrodermal activity (EDA) features and HRV analysis. ECG feature in addition to EDA for affect recognition increases the accuracy significantly. Mood analysis with performing emotion analysis increases affect recognition accuracy significantly. Abstract: Background and objective: Affect provides contextual information about the emotional state of a person as he/she communicates in both verbal and/or non-verbal forms. While human's are great at determining the emotional state of people while they communicate in person, it is challenging and still largely an unsolved problem to computationally determine the emotional state of a person. Methods: Emotional states of a person manifest in the physiological biosignals such as electrocardiogram (ECG) and electrodermal activity (EDA) because these signals are impacted by the peripheral nervous system of the body, and the peripheral nervous system is strongly coupled with the mental state of the person. In this paper, we present a method to accurately recognize six emotions using ECG and EDA signals and applying autoregressive hidden Markov models (AR-HMMs) and heart rate variability analysis on these signals. The six emotions include happiness, sadness, surprise, fear, anger, and disgust. Results: We evaluated our method on a comprehensive new dataset collected from 30 participants. Our results show that our proposed method achieves an average accuracy of 88.6% in distinguishing across the 6 emotions. Conclusions: The key technical depth of the paper is in the use of the AR-HMMs to model the EDA signal and the use of LDA to enable accurate emotion recognition without requiring a large number of training samples. Unlike other studies, we have taken a hierarchical approach to classify emotions, where we first categorize the emotion as either positive or negative and then identify the exact emotion.