A novel online BCI system using speech imagery and ear-EEG for home appliances control. (September 2022)

• Record Type: Journal Article
• Authors: Kaongoen, Netiwit; Choi, Jaehoon; Jo, Sungho
• Is Part Of: Computer methods and programs in biomedicine. Volume 224(2022)
• Publication Date: 2022-09
• Subjects: Brain-computer interface -- Speech-imagery -- Ear-EEG; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2022.107022
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: First online BCI system for control that utilizes speech imagery (SI) via ear-EEG acquired from a custom-made wearable BCI headphone. Temporally-stacked multi-band covariance matrix (TSMBC) method is presented to include temporal, spatial, and spectral information to represent neural activities during SI tasks. The proposed system was used to control an interactive simulated home appliance and tested in both offline and online experiments with eleven participants. Result suggests that the proposed ear-EEG SI-based BCI system is a promising approach for the wearable BCI for daily life. Abstract: Background and Objective: This paper investigates a novel way to interact with home appliances via a brain-computer interface (BCI), using electroencephalograph (EEG) signals acquired from around the user's ears with a custom-made wearable BCI headphone. Methods: The users engage in speech imagery (SI), a type of mental task where they imagine speaking out a specific word without producing any sound, to control an interactive simulated home appliance. In this work, multiple models are employed to improve the performance of the system. Temporally-stacked multi-band covariance matrix (TSMBC) method is used to represent the neural activities during SI tasks with spatial, temporal, and spectral information included. To further increase the usability of our proposed system in daily life, a calibration session, where the pre-trained models are fine-tuned, is added to maintain performance over time with minimal training. Eleven participants were recruited to evaluate our method over three different sessions: a training session, a calibration session, and an online session where users were given the freedom to achieve a given goal on their own. Results: In the offline experiment, all participants were able to achieve a classification accuracy significantly higher than the chance level. In the online experiments, a few participants were able to use the proposed system to freely control the home appliance with high accuracy and relatively fast command delivery speed. The best participant achieved an average true positive rate and command delivery time of 0.85 and 3.79 s/command, respectively. Conclusion: Based on the positive experimental results and user surveys, the novel ear-EEG-SI-based BCI paradigm is a promising approach for the wearable BCI system for daily life.