The Effect of Online and Mixed-Mode Measurement of Cognitive Ability. (February 2019)

• Record Type: Journal Article
• Authors: Al Baghal, Tarek
• Is Part Of: Social science computer review. Volume 37:Number 1(2019)
• Page Start: 89
• Page End: 103
• Publication Date: 2019-02
• Subjects: online surveys -- mixed-mode surveys -- cognitive ability -- self-selection; Social sciences -- Data processing -- Periodicals; Computers -- Social aspects -- Periodicals; Microcomputers -- Periodicals; Sciences sociales -- Informatique -- Périodiques; Micro-ordinateurs -- Périodiques; 300.285
• DOI: 10.1177/0894439317746328
• Languages: English
• ISSNs: 0894-4393

Abstract

A number of studies, particularly longitudinal surveys, are collecting direct measures of cognitive ability, given its importance as a measure in social science research. As longitudinal studies increasingly switch to mixed-mode data collection, frequently including a web component, differences in survey outcomes including cognitive ability may result from mode effects. Differences may arise due to respondent self-selection into mode or due to the mode causing differential measurement. Using a longitudinal survey that measured cognitive ability after introducing a mixed-mode design with a web component, this research explores if and how mode affects cognitive ability outcomes. This survey allows for control of several possible selection mechanisms, including a limited set of direct cognitive ability measures collected in a single mode in an earlier wave. Findings presented here show clearly that web respondents do better on a number of cognitive ability indicators. However, it does not appear that this is wholly explainable by respondents of different ability self-selecting into particular modes. Rather, it appears that measurement of cognitive ability may differ across modes. This result is potentially problematic as comparability is a key component of using cognitive ability in further research.