Identifying fragments of natural speech from the listener's MEG signals. Issue 6 (17th February 2012)

• Record Type: Journal Article
• Authors: Koskinen, Miika; Viinikanoja, Jaakko; Kurimo, Mikko; Klami, Arto; Kaski, Samuel; Hari, Riitta
• Is Part Of: Human brain mapping. Volume 34:Issue 6(2013:Jun.)
• Page Start: 1477
• Page End: 1489
• Publication Date: 2012-02-17
• Subjects: auditory perception -- decoding -- encoding -- machine learning -- magnetoencephalography -- signal processing -- speech; Brain mapping -- Periodicals; 611.81
• DOI: 10.1002/hbm.22004
• Languages: English
• ISSNs: 1065-9471

Abstract

Abstract: It is a challenge for current signal analysis approaches to identify the electrophysiological brain signatures of continuous natural speech that the subject is listening to. To relate magnetoencephalographic (MEG) brain responses to the physical properties of such speech stimuli, we applied canonical correlation analysis (CCA) and a Bayesian mixture of CCA analyzers to extract MEG features related to the speech envelope. Seven healthy adults listened to news for an hour while their brain signals were recorded with whole‐scalp MEG. We found shared signal time series (canonical variates) between the MEG signals and speech envelopes at 0.5–12 Hz. By splitting the test signals into equal‐length fragments from 2 to 65 s (corresponding to 703 down to 21 pieces per the total speech stimulus) we obtained better than chance‐level identification for speech fragments longer than 2–3 s, not used in the model training. The applied analysis approach thus allowed identification of segments of natural speech by means of partial reconstruction of the continuous speech envelope (i.e., the intensity variations of the speech sounds) from MEG responses, provided means to empirically assess the time scales obtainable in speech decoding with the canonical variates, and it demonstrated accurate identification of the heard speech fragments from the MEG data. Hum Brain Mapp, 2013. © 2012 Wiley Periodicals, Inc.