Using Instructed Response Items as Attention Checks in Web Surveys: Properties and Implementation. (February 2021)

• Record Type: Journal Article
• Authors: Gummer, Tobias; Roßmann, Joss; Silber, Henning
• Is Part Of: Sociological methods & research. Volume 50:Number 1(2021)
• Page Start: 238
• Page End: 264
• Publication Date: 2021-02
• Subjects: attention checks -- instructed response items -- grid questions -- measurement error -- data quality -- web surveys; Sociology -- Research -- Periodicals; Sociology -- Methodology -- Periodicals; 301.072
• DOI: 10.1177/0049124118769083
• Languages: English
• ISSNs: 0049-1241

Abstract

Identifying inattentive respondents in self-administered surveys is a challenging goal for survey researchers. Instructed response items (IRIs) provide a measure for inattentiveness in grid questions that is easy to implement. The present article adds to the sparse research on the use and implementation of attention checks by addressing three research objectives. In a first study, we provide evidence that IRIs identify respondents who show an elevated use of straightlining, speeding, item nonresponse, inconsistent answers, and implausible statements throughout a survey. Excluding inattentive respondents, however, did not alter the results of substantive analyses. Our second study suggests that respondents' inattentiveness partially changes as the context in which they complete the survey changes. In a third study, we present experimental evidence that a mere exposure to an IRI does not negatively or positively affect response behavior within a survey. A critical discussion on using IRI attention checks concludes this article.