Capsule neural networks on spatio-temporal EEG frames for cross-subject emotion recognition. (February 2022)

• Record Type: Journal Article
• Authors: Jana, Gopal Chandra; Sabath, Anshuman; Agrawal, Anupam
• Is Part Of: Biomedical signal processing and control. Volume 72(2022)Part B
• Publication Date: 2022-02
• Subjects: Spatio-temporal representation -- Spatio-temporal EEG -- CapsNet -- EEG Emotion Recognition; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.103361
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: Cross-subject emotion recognition study using spatio-temporal frames and CapsNet. For generating spatiotemporal frames, the EEG electrodes' topology is utilized. Novel data-split method to test model with multiple degrees of familiarity is used. Provide a baseline for unseen subject unseen record based emotion classification. Abstract: Scalp EEG plots are plots of scalp potentials against time, and hence, capture spatial information, owing to the placement of electrodes on the scalp, as well as, temporal information from variations in brain waves. In this paper we propose a novel method to make a combined representation of spatial and temporal information, by incorporating the signals into a sparse spatio-temporal frame, such that it can be easily processed by deep learning algorithms in the computer vision domain. Familiarities of a model to the test data in the setting of emotion recognition from EEG, is also defined, and a form of data splitting such that the model has to perform on a set with which it has the minimum degree of familiarity is introduced. A CapsNet architecture is trained on DEAP dataset to perform on a cross-subject binary classification task, and tuning of the hyperparameters using Bayesian Optimization is analyzed. The proposed model reports a best-case accuracy of 0.85396 and average case accuracy of 0.57165 for LOO subject, and a best case of 1.0 and average case of 0.51071 for unseen-subject-unseen-record classification, when averaged across all the classes (i.e., valence, dominance, arousal, and liking), which is comparable to that reported by other works.