The influence of gestures and visuospatial ability during learning about movements with dynamic visualizations – An fNIRS study. (April 2022)

• Record Type: Journal Article
• Authors: Brucker, Birgit; de Koning, Björn; Rosenbaum, David; Ehlis, Ann-Christine; Gerjets, Peter
• Is Part Of: Computers in human behavior. Volume 129(2022)
• Publication Date: 2022-04
• Subjects: Dynamic visualizations -- Gestures -- Learners' visuospatial ability -- Learning about movements -- Human mirror-neuron system -- Functional near-infrared spectroscopy; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2021.107151
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: There is increasing evidence that learning manual tasks from dynamic visualizations (e.g., origami folding) is facilitated when human hands are shown or gestures can be observed in the visualizations. This study examined whether observing and making gestures improves learning about non-human biological movements (i.e., fish locomotion) and whether gestures that correspond to the to-be-learned movement are superior to non-corresponding gestures. Moreover, learners' visuospatial ability was assessed as a possible moderator. Regarding underlying neurophysiological processes, functional near-infrared spectroscopy (fNIRS) was used to investigate whether gestures activate the human mirror-neuron system (hMNS) and whether this activation mediates the facilitation of learning. During learning, participants viewed animations that were supplemented with either a self-gesturing instruction (yes/no) and/or a gesture video (corresponding/non-corresponding/no gesture) resulting in six conditions (2x3-between-subjects design). Results showed that higher-visuospatial-ability learners benefitted from learning with non-corresponding gestures, whereas those gestures were detrimental for lower-visuospatial-ability learners. Furthermore, activation of the inferior frontal cortex (part of hMNS) tended to predict better learning outcomes. Making gestures did not influence learning, but participants observing corresponding gestures showed higher inferior frontal cortex activation if they self-gestured than when they did not self-gesture. Implications of the results for the design of instructional materials are discussed. Highlights: Cognitive and neural effects of gesturing and spatial ability (VSA) were studied in learning from instructional animations. Observing non-corresponding gestures improved learning for higher VSA learners, but reduced learning for lower VSA learners. Activation of the inferior frontal cortex (part of the human mirror-neuron-system) tended to predict better learning. Making gestures did not influence learning directly. When observing corresponding gestures there was higher inferior frontal cortex activation if learners also self-gestured.