Relational local electroencephalography representations for sleep scoring. (October 2022)

• Record Type: Journal Article
• Authors: Brandmayr, Georg; Hartmann, Manfred; Fürbass, Franz; Matz, Gerald; Samwald, Matthias; Kluge, Tilmann; Dorffner, Georg
• Is Part Of: Neural networks. Volume 154(2022)
• Page Start: 310
• Page End: 322
• Publication Date: 2022-10
• Subjects: EEG -- Relative position attention -- Transformer -- Context encoding -- Sequence learning -- REM; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.07.020
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Computational sleep scoring from multimodal neurophysiological time-series (polysomnography PSG) has achieved impressive clinical success. Models that use only a single electroencephalographic (EEG) channel from PSG have not yet received the same clinical recognition, since they lack Rapid Eye Movement (REM) scoring quality. The question whether this lack can be remedied at all remains an important one. We conjecture that predominant Long Short-Term Memory (LSTM) models do not adequately represent distant REM EEG segments (termed epochs), since LSTMs compress these to a fixed-size vector from separate past and future sequences. To this end, we introduce the EEG representation model ENGELBERT (electroEn cephaloG raphic E poch L ocal B idirectional E ncoder R epresentations from T ransformer). It jointly attends to multiple EEG epochs from both past and future. Compared to typical token sequences in language, for which attention models have originally been conceived, overnight EEG sequences easily span more than 1000 30 s epochs. Local attention on overlapping windows reduces the critical quadratic computational complexity to linear, enabling versatile sub-one-hour to all-day scoring. ENGELBERT is at least one order of magnitude smaller than established LSTM models and is easy to train from scratch in a single phase. It surpassed state-of-the-art macro F1-scores in 3 single-EEG sleep scoring experiments. REM F1-scores were pushed to at least 86%. ENGELBERT virtually closed the gap to PSG-based methods from 4–5 percentage points (pp) to less than 1 pp F1-score.