Copula-based transformer in EEG to assess visual discomfort induced by stereoscopic 3D. (August 2022)

• Record Type: Journal Article
• Authors: Zheng, Yawen; Zhao, Xiaojie; Yao, Li
• Is Part Of: Biomedical signal processing and control. Volume 77(2022)
• Publication Date: 2022-08
• Subjects: EEG -- Transformer -- Copula function -- Spatial position dependency -- Temporal dependency -- Visual discomfort; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.103803
• Languages: English
• ISSNs: 1746-8094

Abstract

Graphical abstract: Highlights: Copula-based EEG embedding in CBT integrates the spatial position dependencies with EEG data and converts EEG data into vectors. Copula-based attention in CBT relies on temporal dependencies to extract EEG data that are discriminative and reflect temporal order to reduce the attention matrix's size. L2 norm-based item in the loss function narrows the differences between the extracted and the input EEG to bound the sacrificed information caused by the reduced attention matrix. Abstract: The Transformer deep model (Transformer) has been applied to explore temporal dependencies between electroencephalography (EEG) data or electrode-based spatial connectivity dependencies, which inspires us to introduce it to assess visual discomfort induced by stereoscopic 3D. Many studies utilized the Transformer designed for text directly. EEG data presents (1) electrode-based spatial position dependencies, (2) hierarchical temporal dependencies and (3) limited data volume. However, few studies have focused on spatial position dependencies and the need for the large data volume caused by the attention matrix in Transformer; it is also unclear how they influence visual discomfort assessment. Copula function could model dependencies between variables without strict constraints. This work addresses these gaps by proposing a copula-based Transformer (CBT) to learn spatial position dependencies and reduce the size of the attention matrix to reduce the need for data volume for better visual discomfort assessment. The copula-based EEG embedding module in CBT integrates the spatial position dependencies with EEG data. The copula-based attention module in CBT relies on temporal dependencies to extract EEG data that are discriminative and reflect temporal order to reduce the attention matrix's size. The L2 norm-based item of the loss function narrows the differences between the extracted and the input EEG to bound the sacrificed information caused by the reduced attention matrix. Experiments demonstrate that the accuracy of CBT on visual discomfort assessment could reach 98.08%, which performs best among the listed models. And the extracted EEG data firstly revealed the temporal changes in EEG when visual discomfort arises.