Measuring problem solving skills via stealth assessment in an engaging video game. (October 2016)

• Record Type: Journal Article
• Authors: Shute, Valerie J.; Wang, Lubin; Greiff, Samuel; Zhao, Weinan; Moore, Gregory
• Is Part Of: Computers in human behavior. Volume 63(2016)
• Page Start: 106
• Page End: 117
• Publication Date: 2016-10
• Subjects: Problem solving skills -- Stealth assessment -- Validity; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2016.05.047
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: We used stealth assessment, embedded in a game called Use Your Brainz (a slightly modified version of Plants vs. Zombies 2 ), to measure middle-school students' problem solving skills. We began by developing a problem solving competency model based on a review of the relevant literature. We then identified in-game indicators that would provide evidence about students' levels on various problem-solving facets. Our problem solving model was then implemented in the game via Bayesian networks. To validate the stealth assessment, we collected data from students who played the game-based assessment for three hours and completed two external problem solving measures (i.e., Raven's Progressive Matrices and MicroDYN). Results indicated that the problem solving estimates derived from the game significantly correlated with the external measures, which suggests that our stealth assessment is valid. Our next steps include running a larger validation study and developing tools to help educators interpret the results of the assessment, which will subsequently support the development of problem solving skills. Highlights: We embedded our problem solving competency model in an engaging game. We measured problem solving skills of students via stealth assessment. Two external measures were used for validation: Raven's matrices and MicroDYN. Significant correlations were found between stealth and external measures.