Predicting students' knowledge after playing a serious game based on learning analytics data: A case study. (11th December 2019)

• Record Type: Journal Article
• Authors: Alonso‐Fernández, Cristina; Martínez‐Ortiz, Iván; Caballero, Rafael; Freire, Manuel; Fernández‐Manjón, Baltasar
• Is Part Of: Journal of computer assisted learning. Volume 36:Number 3(2020)
• Page Start: 350
• Page End: 358
• Publication Date: 2019-12-11
• Subjects: learning analytics -- serious games -- game‐based learning -- assessment -- e‐learning -- xAPI; Computer-assisted instruction -- Periodicals; 371.334
• DOI: 10.1111/jcal.12405
• Languages: English
• ISSNs: 0266-4909

Abstract

Abstract: Serious games have proven to be a powerful tool in education to engage, motivate, and help students learn. However, the change in student knowledge after playing games is usually measured with traditional (paper) prequestionnaires–postquestionnaires. We propose a combination of game learning analytics and data mining techniques to predict knowledge change based on in‐game student interactions. We have tested this approach in a case study for which we have conducted preexperiments–postexperiments with 227 students playing a previously validated serious game on first aid techniques. We collected student interaction data while students played, using a game learning analytics infrastructure and the standard data format Experience API for Serious Games. After data collection, we developed and tested prediction models to determine whether knowledge, given as posttest results, can be accurately predicted. Additionally, we compared models both with and without pretest information to determine the importance of previous knowledge when predicting postgame knowledge. The high accuracy of the obtained prediction models suggests that serious games can be used not only to teach but also to measure knowledge acquisition after playing. This will simplify serious games application for educational settings and especially in the classroom easing teachers' evaluation tasks. Lay Description: What is currently known about the subject matter Serious games are a powerful tool to engage, motivate, and help students learn. Pre‐post experiments are commonly used to measure knowledge acquisition. Game learning analytics can be applied to interaction data from games. What this paper adds We present a two‐step approach combining game learning analytics and data mining to predict players' performance in serious games based on their interactions. The approach is tested in a case study with pre‐post experiments collecting interaction data with 227 students playing a serious game to determine if performance can be accurately predicted. The comparison of prediction models has helped to determine if pretest information is essential. The highly accurate prediction models obtained suggest that games can be used to teach and measure knowledge acquisition after playing. Implications of study findings for practitioners The approach aims to simplify the measurement of players' learning with serious games. It may be generalized at least to similar scenarios (e.g., games for procedural learning or game‐likesimulations) where similar interaction data are feasible. Game mechanics and educational design should define the interaction data to capture. Using an accepted standard tracking profile is a clear recommendation.