Digital Game-Based Learning effectiveness assessment: Reflections on study design. (July 2021)

• Record Type: Journal Article
• Authors: All, Anissa; Castellar, Elena Nuñez Patricia; Van Looy, Jan
• Is Part Of: Computers & education. Volume 167(2021)
• Publication Date: 2021-07
• Subjects: Digital game-based learning -- Effectiveness assessment -- Best practices -- Feasibility studies; Education -- Data processing -- Periodicals; Education -- Periodicals; Computers -- Periodicals; Computer-Assisted Instruction -- Periodicals; Éducation -- Informatique -- Périodiques; Electronic journals; 370.285
• DOI: 10.1016/j.compedu.2021.104160
• Languages: English
• ISSNs: 0360-1315

Abstract

Abstract: Despite efforts for defining best practices for assessing Digital Game-Based Learning (DGBL) effectiveness, lack of scientific rigor is yet to be established. This has led to academics and educational practitioners doubting the quality of evidence and practical value of scientific research in educational settings. Hence, the present manuscript aims to test the feasibility of previously defined best practices by means of 3 feasibility studies: one in formal, one in health and one in corporate education. Results firstly show a more nuanced view on previously defined best practices regarding control groups: a) inclusion of an educational activity is not always desirable and depends on whether absolute or relative effectiveness is assessed and b) keeping instructional time equal in experimental and control group does not comply with the time efficiency outcome of DGBL. Secondly, several non-intervention related elements jeopardizing internal validity have been established: a) failed randomization, which can be tackled with blocked randomization and b) pre-test effects, which can be tackled with carefully piloted parallel versions of tests. Lastly, additional indicators for motivational and efficiency outcomes in a self-paced distance learning context have been established: a) motivation during/after the training should be expanded with motivation to start playing and b) time required to finish the training should be expended with time required to follow-up on learners. Recommendations in the present manuscript are not exhaustive or generalizable to all contexts, but do provide preliminary insights into feasible experimental designs for DGBL effectiveness studies. Highlights: Tree studies testing the feasibility of experimental design components for DGBL effectiveness. Distinction between relative and absolute effectiveness impacts research design. Educational activity in control group not always desirable. Motivational and efficiency dimension impacts desirability of equal time-on-task. Recommendations for practical constraints regarding optimal research designs.