Visuospatial Working Memory and Understanding Co-Speech Iconic Gestures: Do Gestures Help to Paint a Mental Picture?. Issue 4 (21st April 2022)

• Record Type: Journal Article
• Authors: Wu, Ying Choon; Müller, Horst M.; Coulson, Seana
• Is Part Of: Discourse processes. Volume 59:Issue 4(2022)
• Page Start: 275
• Page End: 297
• Publication Date: 2022-04-21
• Subjects: Discourse analysis -- Periodicals; 401.41
• DOI: 10.1080/0163853X.2022.2028087
• Languages: English
• ISSNs: 0163-853X

Abstract

ABSTRACT: Multi-modal discourse comprehension requires speakers to combine information from speech and gestures. To date, little research has addressed the cognitive resources that underlie these processes. Here we used a dual-task paradigm to test the relative importance of verbal and visuospatial working memory in speech-gesture comprehension. Healthy, college-aged participants encoded either a series of digits (verbal load) or a series of dot locations in a grid (visuospatial load) and rehearsed them (secondary memory task) as they performed a (primary) multi-modal discourse comprehension task. Regardless of the secondary task, performance on the discourse comprehension task was better when the speaker's gestures and speech were congruent than when they were incongruent. However, the congruity advantage was smaller when the concurrent memory task involved a visuospatial load than when it involved a verbal load. Results suggest that taxing the visuospatial working memory system reduced participants' ability to benefit from the information in congruent iconic gestures. A control experiment demonstrated that results were not an artifact of the difficulty of the visuospatial load task. Overall, these data suggest speakers recruit visuospatial working memory to interpret gestures about concrete visual scenes.