A multiple frequency bands parallel spatial–temporal 3D deep residual learning framework for EEG-based emotion recognition. (January 2023)

• Record Type: Journal Article
• Authors: Miao, Minmin; Zheng, Longxin; Xu, Baoguo; Yang, Zhong; Hu, Wenjun
• Is Part Of: Biomedical signal processing and control. Volume 79(2023)Part 2
• Publication Date: 2023-01
• Subjects: Emotion recognition -- Brain-computer interface -- Neural signatures -- Frequency-spatial–temporal; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.104141
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: Electroencephalography (EEG) based emotion recognition has become a hot research issue in the field of cognitive interaction and brain-computer interface (BCI). How to build a deep learning model which can fully learn frequency-spatial–temporal representation from complex emotional EEG data and has good neurological interpretability is still challenging. In this paper, a novel multiple frequency bands parallel spatial–temporal 3D deep residual learning framework (MFBPST-3D-DRLF) is proposed for EEG-based emotion recognition. Firstly, a new optimal frequency bands selection method based on group sparse regression is designed for characteristic analysis on frequency domain. Secondly, spatial–temporal 3D feature representations of multiple frequency bands are generated in the data preparation stage for fully expressing the discriminative local patterns among brain responses of different emotional states. Finally, a novel parallel 3D deep residual networks architecture is elaborately constructed to simultaneously extract high level abstract features and achieve accurate classification. Emotional EEG recognition performance of the proposed method has been evaluated on two benchmark datasets, namely SEED and SEED-IV. The proposed MFBPST-3D-DRLF achieves 96.67% and 88.21% on both datasets, outperforming several state-of-the-art algorithms. In addition, investigations on the intermediate results and model parameters reveal that neural signatures associated with different emotional states are traceable and gamma band is most suitable for EEG based emotion recognition. Graphical abstract: Highlights: MFBPST-3D-DRLF has good representation learning ability on entire frequency-spatial–temporal domains. MFBPST-3D-DRLF outperforms state-of-the-art algorithms on both SEED and SEED-IV datasets. Single gamma band is most suitable for classifying both three-class and four-class emotions. Neural signatures associated with different emotional states are traceable.