Recognition of emotional states using frequency effective connectivity maps through transfer learning approach from electroencephalogram signals. (May 2022)

• Record Type: Journal Article
• Authors: Bagherzadeh, Sara; Maghooli, Keivan; Shalbaf, Ahmad; Maghsoudi, Arash
• Is Part Of: Biomedical signal processing and control. Volume 75(2022)
• Publication Date: 2022-05
• Subjects: Emotion Recognition (ER) -- Electroencephalogram (EEG) -- Effective Connectivity (EC) -- Partial Directed Coherence (PDC) -- Convolutional Neural Network (CNN) -- Transfer Learning (TL); Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.103544
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: Convolutional Neural Networks (CNN) have been widely utilized in Emotion Recognition (ER) research due to their vast benefits. Designing specific CNN configuration and learning accurate parameters are provided through the Transfer Learning (TL) approach. A novel combined ER schema based on sophisticated Frequency Effective Connectivity (FEC) maps and transfer learned CNN models named FEC-CNN is proposed in this study. The EC measure estimates the information flow between brain channels in frequency bands regarding the volume conduction problem. Then, visualization of this information provides valuable and distinctive information for classifying emotional states. After preprocessing Electroencephalogram (EEG) signals, the Effective Connectivity maps were computed through the Partial Directed Coherence (PDC) measure from EEG channels at standard delta, theta, alpha, beta, and gamma frequency bands. Then, PDC maps were used to fine-tune six pre-trained CNN models, AlexNet, ResNet-18, DarkNet-19, ShuffleNet, Inception-v3, and Xception, and classify five emotional states toward the subject independent Leave-One-Subject-Out (LOSO) Cross-Validation criterion for MAHNOB-HCI, DEAP and DREAMER databases. The experimental results show that ResNet-18 achieved the highest average accuracy and F-score 94.27%, 94.74% for the DEAP database, and 95.25% and 95.33% for the MAHNOB-HCI database, and 96.00% and 96.77% for DREAMER database, to recognize five emotional states from PDC maps at alpha frequency bands. Comparison of the experimental results with recent researches showed the superiority of the proposed EC-based CNN model. This research introduced an interesting sight of deep learning applications in cognitive computation area by utilizing effective connectivity information flows to recognize emotional states from EEG signals.