Effects of picture-word integration on reading visual narratives in L1 and L2. (February 2021)

• Record Type: Journal Article
• Authors: Yum, Yen Na; Cohn, Neil; Lau, Way Kwok-Wai
• Is Part Of: Learning and instruction. Volume 71(2021)
• Publication Date: 2021-02
• Subjects: Bilingual -- Multimodal reading -- Perceived ease -- Situational interest -- Visual narratives; Learning -- Periodicals; Teaching -- Periodicals; Apprentissage -- Périodiques; Enseignement -- Périodiques; Learning; Teaching; Periodicals; Electronic journals; 370.1
• DOI: 10.1016/j.learninstruc.2020.101397
• Languages: English
• ISSNs: 0959-4752

Abstract

Abstract: Multimodal education materials are pervasive in language learning. This study investigated the causal mechanisms of multimodal reading effects in first language (L1) and second language (L2). Seventy-five adult bilingual readers in Hong Kong read Chinese and English passages with different degrees of picture-word integration in a within-subject design. Results showed that tight text-picture integration facilitated better comprehension than independent text-picture presentation in L2, but not L1. Perceived ease and interest differentially mediated multimodal reading performance for L1 and L2 passages. Importantly, separate images in L2 passages led to poorer comprehension accuracy relative to plain text, but tended to have higher ratings of ease and interest, indicating that readers may be overconfident in their multimodal reading performance. In general, results support the notion that integration of text and pictures can moderate the process of meaning making, and these may differ depending on the language presented to a bilingual reader. Highlights: Text-image integration impacted the reading performance of adult bilinguals. Multimodal effects were mediated by situational interest and perceived ease. Presentation mode and language interacted in multimodal reading. Readers showed overconfidence when reading multimodal materials in L2.