An electrophysiological investigation of co-referential processes in visual narrative comprehension. (29th July 2022)

• Record Type: Journal Article
• Authors: Coopmans, Cas W.; Cohn, Neil
• Is Part Of: Neuropsychologia. Number 172(2022)
• Publication Date: 2022-07-29
• Subjects: Comics -- Anaphora -- ERPs -- Oscillations -- Nref -- Alpha; Neuropsychology -- Periodicals; Neurology -- Periodicals; Psychophysiology -- Periodicals; Neuropsychologie -- Périodiques; Neuropsychology; Periodicals; Electronic journals; 616.8
• DOI: 10.1016/j.neuropsychologia.2022.108253
• Languages: English
• ISSNs: 0028-3932

Abstract

Abstract: Visual narratives make use of various means to convey referential and co-referential meaning, so comprehenders must recognize that different depictions across sequential images represent the same character(s). In this study, we investigated how the order in which different types of panels in visual sequences are presented affects how the unfolding narrative is comprehended. Participants viewed short comic strips while their electroencephalogram (EEG) was recorded. We analyzed evoked and induced EEG activity elicited by both full panels (showing a full character) and refiner panels (showing only a zoom of that full panel), and took into account whether they preceded or followed the panel to which they were co-referentially related (i.e., were cataphoric or anaphoric). We found that full panels elicited both larger N300 amplitude and increased gamma-band power compared to refiner panels. Anaphoric panels elicited a sustained negativity compared to cataphoric panels, which appeared to be sensitive to the referential status of the anaphoric panel. In the time-frequency domain, anaphoric panels elicited reduced 8–12 Hz alpha power and increased 45–65 Hz gamma-band power compared to cataphoric panels. These findings are consistent with models in which the processes involved in visual narrative comprehension partially overlap with those in language comprehension. Highlights: Visual narratives require comprehenders to recognize co-reference across panels EEG activity dissociates effects of anaphoricity and panel type in visual narratives Panel type was associated with modulations of N300 amplitude and gamma power Anaphoric panels elicited an increased frontal negativity and decreased alpha power Co-referential processes are similar in visual narratives and language