The neural representations underlying asymmetric cross‐modal prediction of words. Issue 6 (30th January 2023)

• Record Type: Journal Article
• Authors: Shi, Liang; Liu, Chuqi; Peng, Xiaojing; Cao, Yifei; Levy, Daniel A.; Xue, Gui
• Is Part Of: Human brain mapping. Volume 44:Issue 6(2023)
• Page Start: 2418
• Page End: 2435
• Publication Date: 2023-01-30
• Subjects: cross‐modal prediction -- fMRI -- representation -- sequence memory -- word sequence; Brain mapping -- Periodicals; 611.81
• DOI: 10.1002/hbm.26219
• Languages: English
• ISSNs: 1065-9471

Abstract

Abstract: Cross‐modal prediction serves a crucial adaptive role in the multisensory world, yet the neural mechanisms underlying this prediction are poorly understood. The present study addressed this important question by combining a novel audiovisual sequence memory task, functional magnetic resonance imaging (fMRI), and multivariate neural representational analyses. Our behavioral results revealed a reliable asymmetric cross‐modal predictive effect, with a stronger prediction from visual to auditory (VA) modality than auditory to visual (AV) modality. Mirroring the behavioral pattern, we found the superior parietal lobe (SPL) showed higher pattern similarity for VA than AV pairs, and the strength of the predictive coding in the SPL was positively correlated with the behavioral predictive effect in the VA condition. Representational connectivity analyses further revealed that the SPL mediated the neural pathway from the visual to the auditory cortex in the VA condition but was not involved in the auditory to visual cortex pathway in the AV condition. Direct neural pathways within the unimodal regions were found for the visual‐to‐visual and auditory‐to‐auditory predictions. Together, these results provide novel insights into the neural mechanisms underlying cross‐modal sequence prediction. Abstract : Visual–auditory (VA) prediction was stronger than auditory–visual (AV) prediction. Superior parietal lobe (SPL) shows higher neural similarity for VA than AV prediction. Indirect pathway from the visual to auditory cortex via SPL underlies VA prediction. In contrast, direct connectivity within modality‐specific areas supports within‐modal predictions.