Assessing the validity of inferences from scores on the cognitive reflection test. (7th June 2019)

• Record Type: Journal Article
• Authors: Blacksmith, Nikki; Yang, Yongwei; Behrend, Tara S.; Ruark, Gregory A.
• Is Part Of: Journal of behavioral decision making. Volume 32:Number 5(2019)
• Page Start: 599
• Page End: 612
• Publication Date: 2019-06-07
• Subjects: bias -- cognitive reflection -- decision making -- heuristics -- item response theory; Decision making -- Periodicals; Managerial economics -- Periodicals; Behavior -- Periodicals; Decision Making -- Periodicals; Prise de décision -- Périodiques; Économie d'entreprise -- Périodiques; 153.83
• DOI: 10.1002/bdm.2133
• Languages: English
• ISSNs: 0894-3257

Abstract

Abstract: Decision‐making researchers purport that a novel cognitive ability construct, cognitive reflection, explains variance in intuitive thinking processes that traditional mental ability constructs do not. However, researchers have questioned the validity of the primary measure because of poor construct conceptualization and lack of validity studies. Prior studies have not adequately aligned the analytical techniques with the theoretical basis of the construct, dual‐processing theory of reasoning. The present study assessed the validity of inferences drawn from the cognitive reflection test (CRT) scores. We analyzed response processes with an item response tree model, a method that aligns with the dual‐processing theory in order to interpret CRT scores. Findings indicate that the intuitive and reflective factors that the test purportedly measures were indistinguishable. Exploratory, post hoc analyses demonstrate that CRT scores are most likely capturing mental abilities. We suggest that future researchers recognize and distinguish between individual differences in cognitive abilities and cognitive processes.