Temporal-spatial-frequency depth extraction of brain-computer interface based on mental tasks. (April 2020)

• Record Type: Journal Article
• Authors: Wang, Li; Huang, Weijian; Yang, Zhao; Zhang, Chun
• Is Part Of: Biomedical signal processing and control. Volume 58(2020)
• Publication Date: 2020-04
• Subjects: Brain-computer interface (BCI) -- Electroencephalogram (EEG) -- Temporal-spatial-frequency -- Convolutional neural network (CNN) -- Long short term memory (LSTM); Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2020.101845
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: Series and parallel structures are proposed by combining CNN and LSTM. Temporal-spatial-frequency features of EEG are extracted at the same time. Filters of the convolution layer are visualizd to interpret and understand CNN. The series structure with compact CNN obtains the best result. Abstract: With the help of brain-computer interface (BCI) systems, the electroencephalography (EEG) signals can be translated into control commands. It is rare to extract temporal-spatial-frequency features of the EEG signals at the same time by conventional deep neural networks. In this study, two types of series and parallel structures are proposed by combining convolutional neural network (CNN) and long short term memory (LSTM). The frequency and spatial features of EEG are extracted by CNN, and the temporal features are extracted by LSTM. The EEG signals of mental tasks with speech imagery are extracted and classified by these architectures. In addition, the proposed methods are further validated by the 2008 BCI competition IV-2a EEG data set, and its mental task is motor imagery. The series structure with compact CNN obtains the best results for two data sets. Compared with the algorithms of other literatures, our proposed method achieves the best result. Better classification results can be obtained by designing the well structured deep neural network.