Neurophysiology underlying influence of stimulus reliability on audiovisual integration. (9th February 2018)

• Record Type: Journal Article
• Authors: Shatzer, Hannah; Shen, Stanley; Kerlin, Jess R.; Pitt, Mark A.; Shahin, Antoine J.
• Other Names: Mazaheri Ali guestEditor.; Slagter Heleen guestEditor.; Thut Gregor guestEditor.; Foxe John J. guestEditor.
• Is Part Of: European journal of neuroscience. Volume 48:Number 8(2018)
• Page Start: 2836
• Page End: 2848
• Publication Date: 2018-02-09
• Subjects: alpha oscillations -- audiovisual asynchrony -- audiovisual integration -- event‐related potentials; Nervous system -- Periodicals; 612.8
• DOI: 10.1111/ejn.13843
• Languages: English
• ISSNs: 0953-816X

Abstract

Abstract: We tested the predictions of the dynamic reweighting model (DRM) of audiovisual (AV) speech integration, which posits that spectrotemporally reliable (informative) AV speech stimuli induce a reweighting of processing from low‐level to high‐level auditory networks. This reweighting decreases sensitivity to acoustic onsets and in turn increases tolerance to AV onset asynchronies (AVOA). EEG was recorded while subjects watched videos of a speaker uttering trisyllabic nonwords that varied in spectrotemporal reliability and asynchrony of the visual and auditory inputs. Subjects judged the stimuli as in‐sync or out‐of‐sync. Results showed that subjects exhibited greater AVOA tolerance for non‐blurred than blurred visual speech and for less than more degraded acoustic speech. Increased AVOA tolerance was reflected in reduced amplitude of the P1‐P2 auditory evoked potentials, a neurophysiological indication of reduced sensitivity to acoustic onsets and successful AV integration. There was also sustained visual alpha band (8–14 Hz) suppression (desynchronization) following acoustic speech onsets for non‐blurred vs. blurred visual speech, consistent with continuous engagement of the visual system as the speech unfolds. The current findings suggest that increased spectrotemporal reliability of acoustic and visual speech promotes robust AV integration, partly by suppressing sensitivity to acoustic onsets, in support of the DRM's reweighting mechanism. Increased visual signal reliability also sustains the engagement of the visual system with the auditory system to maintain alignment of information across modalities. Abstract : This study investigated the relationship between stimulus quality and tolerance for audiovisual onset asynchrony (AVOA). Greater AVOA tolerance was observed for more reliable speech signals, which was accompanied by reduction in the P1‐P2 auditory‐evoked potentials for non‐blurred vs. blurred visual speech, and for less vs. more degraded acoustic speech. Hence, increased stimulus spectrotemporal reliability facilitates AV integration by weakening early activity at the auditory cortex.