Electrophysiological evidence for Audio-visuo-lingual speech integration. (31st January 2018)

• Record Type: Journal Article
• Authors: Treille, Avril; Vilain, Coriandre; Schwartz, Jean-Luc; Hueber, Thomas; Sato, Marc
• Is Part Of: Neuropsychologia. Volume 109(2018)
• Page Start: 126
• Page End: 133
• Publication Date: 2018-01-31
• Subjects: Speech perception -- Audio-visual integration -- Visual tongue feedback -- Action perception -- EEG; Neuropsychology -- Periodicals; Neurology -- Periodicals; Psychophysiology -- Periodicals; Neuropsychologie -- Périodiques; Neuropsychology; Periodicals; Electronic journals; 616.8
• DOI: 10.1016/j.neuropsychologia.2017.12.024
• Languages: English
• ISSNs: 0028-3932

Abstract

Abstract: Recent neurophysiological studies demonstrate that audio-visual speech integration partly operates through temporal expectations and speech-specific predictions. From these results, one common view is that the binding of auditory and visual, lipread, speech cues relies on their joint probability and prior associative audio-visual experience. The present EEG study examined whether visual tongue movements integrate with relevant speech sounds, despite little associative audio-visual experience between the two modalities. A second objective was to determine possible similarities and differences of audio-visual speech integration between unusual audio-visuo-lingual and classical audio-visuo-labial modalities. To this aim, participants were presented with auditory, visual, and audio-visual isolated syllables, with the visual presentation related to either a sagittal view of the tongue movements or a facial view of the lip movements of a speaker, with lingual and facial movements previously recorded by an ultrasound imaging system and a video camera. In line with previous EEG studies, our results revealed an amplitude decrease and a latency facilitation of P2 auditory evoked potentials in both audio-visual-lingual and audio-visuo-labial conditions compared to the sum of unimodal conditions. These results argue against the view that auditory and visual speech cues solely integrate based on prior associative audio-visual perceptual experience. Rather, they suggest that dynamic and phonetic informational cues are sharable across sensory modalities, possibly through a cross-modal transfer of implicit articulatory motor knowledge. Highlights: The EEG study examined whether visual tongue movements integrate with speech sounds. A modulation of P2 AEPs was observed for AV compared to A+V conditions. Dynamic and phonetic informational cues appear sharable across sensory modalities.