Getting a cue before getting a clue: Event-related potentials to inference in visual narrative comprehension. (October 2015)

• Record Type: Journal Article
• Authors: Cohn, Neil; Kutas, Marta
• Is Part Of: Neuropsychologia. Volume 77(2015)
• Page Start: 267
• Page End: 278
• Publication Date: 2015-10
• Subjects: Visual language -- Comics -- Narrative -- Visual Narrative Grammar -- Discourse -- Inference -- Event-related potentials; Neuropsychology -- Periodicals; Neurology -- Periodicals; Psychophysiology -- Periodicals; Neuropsychologie -- Périodiques; Neuropsychology; Periodicals; Electronic journals; 616.8
• DOI: 10.1016/j.neuropsychologia.2015.08.026
• Languages: English
• ISSNs: 0028-3932

Abstract

Abstract: Inference has long been emphasized in the comprehension of verbal and visual narratives. Here, we measured event-related brain potentials to visual sequences designed to elicit inferential processing. In Impoverished sequences, an expressionless "onlooker" watches an undepicted event (e.g., person throws a ball for a dog, then watches the dog chase it) just prior to a surprising finale (e.g., someone else returns the ball), which should lead to an inference (i.e., the different person retrieved the ball). Implied sequences alter this narrative structure by adding visual cues to the critical panel such as a surprised facial expression to the onlooker implying they saw an unexpected, albeit undepicted, event. In contrast, Expected sequences show a predictable, but then confounded, event (i.e., dog retrieves ball, then different person returns it), and Explicit sequences depict the unexpected event (i.e., different person retrieves then returns ball). At the critical penultimate panel, sequences representing depicted events (Explicit, Expected) elicited a larger posterior positivity (P600) than the relatively passive events of an onlooker (Impoverished, Implied), though Implied sequences were slightly more positive than Impoverished sequences. At the subsequent and final panel, a posterior positivity (P600) was greater to images in Impoverished sequences than those in Explicit and Implied sequences, which did not differ. In addition, both sequence types requiring inference (Implied, Impoverished) elicited a larger frontal negativity than those explicitly depicting events (Expected, Explicit). These results show that neural processing differs for visual narratives omitting events versus those depicting events, and that the presence of subtle visual cues can modulate such effects presumably by altering narrative structure. Highlights: We measured ERPs to visual narratives involving inference generation. Greater change in event structure elicited P600s at critical and subsequent panels. Anterior negativities arose for visual narratives omitting events versus depicting events. Processing differed for sequences using inference but varying in narrative structure. Subtle visual cues can modulate both inferential and narrative processing.