Electrophysiological Indexes of Incongruent Audiovisual Phonemic Processing: Unraveling the McGurk Effect. (10th August 2018)

• Record Type: Journal Article
• Authors: Proverbio, Alice Mado; Raso, Giulia; Zani, Alberto
• Is Part Of: Neuroscience. Volume 385(2018)
• Page Start: 215
• Page End: 226
• Publication Date: 2018-08-10
• Subjects: AV audiovisual -- BEM boundary element model -- ERP evoked response potential -- IFG inferior frontal gyrus -- LORETA Low-Resolution Electromagnetic Tomography -- MTG medial temporal gyrus -- pMMN phonological Mismatch Negativity -- PMv ventral premotor -- POp pars opercularis -- pSTS posterior superior temporal sulcus -- SMG supramarginal gyrus -- STG superior temporal -- STp posterior superior temporal -- STS superior temporal sulcus -- TVSA temporal visual speech area; language -- ERP -- illusion -- audiovisual -- incongruence -- MMN; Neurochemistry -- Periodicals; Neurophysiology -- Periodicals; Neurology -- Periodicals; Neurochimie -- Périodiques; Neurophysiologie -- Périodiques; Neurochemistry; Neurophysiology; Electronic journals; Periodicals; Electronic journals; 612.8
• DOI: 10.1016/j.neuroscience.2018.06.021
• Languages: English
• ISSNs: 0306-4522

Abstract

Graphical abstract: Highlights: EEG/ERPs were recorded in 14 Italian speakers while perceiving congruent or incongruent audiovisual (AV) stimuli. AV incongruent pairs producing optimal McGurk illusions were presented. A pMMN (phonological Mismatch Negativity) to incongruent AV stimuli was recorded. Left hemispheric temporo/parietal areas supported phoneme categorization. The right STS (involved in AV integration) was at the basis of perceptual illusion. Abstract: In this study the timing of electromagnetic signals recorded during incongruent and congruent audiovisual (AV) stimulation in 14 Italian healthy volunteers was examined. In a previous study (Proverbio et al., 2016) we investigated the McGurk effect in the Italian language and found out which visual and auditory inputs provided the most compelling illusory effects (e.g., bilabial phonemes presented acoustically and paired with non-labials, especially alveolar–nasal and velar–occlusive phonemes). In this study EEG was recorded from 128 scalp sites while participants observed a female and a male actor uttering 288 syllables selected on the basis of the previous investigation (lasting approximately 600 ms) and responded to rare targets (/re/, /ri/, /ro/, /ru/). In half of the cases the AV information was incongruent, except for targets that were always congruent. A pMMN (phonological Mismatch Negativity) to incongruent AV stimuli was identified 500 ms after voice onset time. This automatic response indexed the detection of an incongruity between the labial and phonetic information. SwLORETA (Low-Resolution Electromagnetic Tomography) analysis applied to the difference voltage incongruent–congruent in the same time window revealed that the strongest sources of this activity were the right superior temporal (STG) and superior frontal gyri, which supports their involvement in AV integration.