Home alone: Complex problem solving performance benefits from individual online assessment. (March 2017)

• Record Type: Journal Article
• Authors: Schult, Johannes; Stadler, Matthias; Becker, Nicolas; Greiff, Samuel; Sparfeldt, Jörn R.
• Is Part Of: Computers in human behavior. Volume 68(2017)
• Page Start: 513
• Page End: 519
• Publication Date: 2017-03
• Subjects: Complex problem solving -- Computer-based assessment -- Individual administration -- Group testing -- Online testing; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2016.11.054
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Computer-based assessments of complex problem solving performance often take place in group settings like classrooms and computer laboratories. Such computer-based procedures provide an excellent opportunity to examine setting effects that might occur while participants are tested in a non-group session online at a time and place of their own choosing. For this purpose, N = 273 teacher students were randomly assigned to one of two settings: the individual online condition ( n = 216) or the computer laboratory group condition ( n = 57). Strong factorial measurement invariance was evidenced. Participants performed significantly better in the individual online condition than in the group condition (knowledge acquisition: d = 0.38; knowledge application: d = 0.39). The worse performance in the group setting compared to the individual setting could neither be explained by exploration time, nor by time on task. The internal experimental design validity strengthens the conclusion that setting-related differences in cognitive ability testing are not negligible but noteworthy. Highlights: Complex problem solving performance was assessed in two computer-based settings. Settings differed in terms of location, presence of others, and schedule. Individual online performance was slightly better than laboratory group performance. Differences in time on task cannot explain performance differences. Differences in exploration time cannot explain performance differences.