Using virtual presence and survey instructions to minimize careless responding on Internet-based surveys. (July 2015)

• Record Type: Journal Article
• Authors: Ward, M.K.; Pond, Samuel B.
• Is Part Of: Computers in human behavior. Volume 48(2015)
• Page Start: 554
• Page End: 568
• Publication Date: 2015-07
• Subjects: Online survey -- Survey design -- Careless responding -- Insufficient effort responding -- Virtual presence -- Instructions; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2015.01.070
• Languages: English
• ISSNs: 0747-5632

Abstract

Highlights: Instructions and virtual presence were used to prevent careless responding in online surveys. MANOVAs showed warning instructions reduced some forms of careless responding. Virtual presence enhances the utility of survey instructions in preventing careless responding. Feedback instructions hold promise as a non-aversive way to prevent careless responding. Abstract: Internet-based survey data inform knowledge creation in research and justify work activities in organizations. While there are advantages to online surveys, this mode of administration comes with its own set of challenges. Survey respondents may engage in careless responding (i.e. insufficient effort responding or satisficing) by intentionally or unintentionally responding in a manner that does not accurately reflect their true sentiments. Careless responding can create psychometric problems even after correctly removing careless respondents (i.e. mischievous responders). This study aimed to improve survey methodology by preventing careless responding. Using a 3 × 3 between-subjects experimental design, we manipulated both virtual presence (none, animated shape, and virtual human) and type of instructions (anonymous, warning, and feedback). Indicators of careless responding were the dependent variables. Results showed that beyond characteristics of survey items, survey design elements can prevent careless responding. The effects of interventions differed by type of careless responding. Instructions, and the interaction of instructions and virtual presence significantly reduced careless responding, but not virtual presence alone. Virtual human presence increased the salience of instructions. Although currently effective, instructions warning of punitive consequences may create difficulty in recruiting participants. Future research should continue investigating non-aversive ways to prevent careless responding on Internet-based surveys.