Electrophysiological Indexes of Incongruent Audiovisual Phonemic Processing: Unraveling the McGurk Effect. (10th August 2018) - Proverbio, Alice Mado; Raso, Giulia; Zani, Alberto
Graphical abstract: Highlights: EEG/ERPs were recorded in 14 Italian speakers while perceiving congruent or incongruent audiovisual (AV) stimuli. AV incongruent pairs producing optimal McGurk illusions were presented. A pMMN (phonological Mismatch Negativity) to incongruent AV stimuli was recorded. Left hemispheric temporo/parietal areas supported phoneme categorization. The right STS (involved in AV integration) was at the basis of perceptual illusion. Abstract: In this study the timing of electromagnetic signals recorded during incongruent and congruent audiovisual (AV) stimulation in 14 Italian healthy volunteers was examined. In a previous study (Proverbio et al., 2016) we investigated the McGurk effect in the Italian language and found out which visual and auditory inputs provided the most compelling illusory effects (e.g., bilabial phonemes presented acoustically and paired with non-labials, especially alveolar–nasal and velar–occlusive phonemes). In this study EEG was recorded from 128 scalp sites while participants observed a female and a male actor uttering 288 syllables selected on the basis of the previous investigation (lasting approximately 600 ms) and responded to rare targets (/re/, /ri/, /ro/, /ru/). In half of the cases the AV information was incongruent, except for targets that were always congruent. A pMMN (phonological Mismatch Negativity) to incongruent AV stimuli was identified 500 ms after voice onset time. This automatic response indexed the detection of an incongruity between the labial and phonetic information. SwLORETA (Low-Resolution Electromagnetic Tomography) analysis applied to the difference voltage incongruent–congruent in the same time window revealed that the strongest sources of this activity were the right superior temporal (STG) and superior frontal gyri, which supports their involvement in AV integration.