Evaluating non-affective cross-modal congruence effects on emotion perception. Issue 8 (17th November 2021)

• Record Type: Journal Article
• Authors: Gao, Chuanji; Wedell, Douglas H.; Shinkareva, Svetlana V.
• Is Part Of: Cognition and emotion. Volume 35:Issue 8(2021)
• Page Start: 1634
• Page End: 1651
• Publication Date: 2021-11-17
• Subjects: Multisensory -- emotion -- affect -- audiovisual integration -- cross-modal congruence; Cognition -- Periodicals; Emotions and cognition -- Periodicals; 155.413
• DOI: 10.1080/02699931.2021.1973966
• Languages: English
• ISSNs: 0269-9931

Abstract

ABSTRACT: Although numerous studies have shown that people are more likely to integrate consistent visual and auditory signals, the role of non-affective congruence in emotion perception is unclear. This registered report examined the influence of non-affective cross-modal congruence on emotion perception. In Experiment 1, non-affective congruence was manipulated by matching or mismatching gender between visual and auditory modalities. Participants were instructed to attend to emotion information from only one modality while ignoring the other modality. Experiment 2 tested the inverse effectiveness rule by including both noise and noiseless conditions. Across two experiments, we found the effects of task-irrelevant emotional signals from one modality on emotional perception in the other modality, reflected in affective congruence, facilitation, and affective incongruence effects. The effects were stronger for the attend-auditory compared to the attend-visual condition, supporting a visual dominance effect. The effects were stronger for the noise compared to the noiseless condition, consistent with the inverse effectiveness rule. We did not find evidence for the effects of non-affective congruence on audiovisual integration of emotion across two experiments, suggesting that audiovisual integration of emotion may not require automatic integration of non-affective congruence information.