Single‐Stage Prediction Models Do Not Explain the Magnitude of Syntactic Disambiguation Difficulty. Issue 6 (25th June 2021)

• Record Type: Journal Article
• Authors: van Schijndel, Marten; Linzen, Tal
• Is Part Of: Cognitive science. Volume 45:Issue 6(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-06-25
• Subjects: Self‐paced reading -- Garden paths -- Neural networks -- Surprisal -- Information theory; Cognition -- Periodicals; Psycholinguistics -- Periodicals; Artificial intelligence -- Periodicals; 153.05
• DOI: 10.1111/cogs.12988
• Languages: English
• ISSNs: 0364-0213

Abstract

Abstract: The disambiguation of a syntactically ambiguous sentence in favor of a less preferred parse can lead to slower reading at the disambiguation point. This phenomenon, referred to as a garden‐path effect, has motivated models in which readers initially maintain only a subset of the possible parses of the sentence, and subsequently require time‐consuming reanalysis to reconstruct a discarded parse. A more recent proposal argues that the garden‐path effect can be reduced to surprisal arising in a fully parallel parser: words consistent with the initially dispreferred but ultimately correct parse are simply less predictable than those consistent with the incorrect parse. Since predictability has pervasive effects in reading far beyond garden‐path sentences, this account, which dispenses with reanalysis mechanisms, is more parsimonious. Crucially, it predicts a linear effect of surprisal: the garden‐path effect is expected to be proportional to the difference in word surprisal between the ultimately correct and ultimately incorrect interpretations. To test this prediction, we used recurrent neural network language models to estimate word‐by‐word surprisal for three temporarily ambiguous constructions. We then estimated the slowdown attributed to each bit of surprisal from human self‐paced reading times, and used that quantity to predict syntactic disambiguation difficulty. Surprisal successfully predicted the existence of garden‐path effects, but drastically underpredicted their magnitude, and failed to predict their relative severity across constructions. We conclude that a full explanation of syntactic disambiguation difficulty may require recovery mechanisms beyond predictability.