Quantifying teaching quality in medical education: The impact of learning gain calculation. Issue 3 (2nd December 2021)

• Record Type: Journal Article
• Authors: Westphale, Silke; Backhaus, Joy; Koenig, Sarah
• Is Part Of: Medical education. Volume 56:Issue 3(2022)
• Page Start: 312
• Page End: 320
• Publication Date: 2021-12-02
• Subjects: Medical education -- Periodicals; Medical education -- Great Britain -- Periodicals; 610.7
• DOI: 10.1111/medu.14694
• Languages: English
• ISSNs: 0308-0110

Abstract

Abstract: Background: Student performance is a mirror of teaching quality. The pre‐/post‐test design allows a pragmatic approach to comparing the effects of interventions. However, the calculation of current knowledge gain scores introduces varying degrees of distortion. Here we present a new metric employing a linear weighting coefficient to reduce skewness on outcome interpretation. Methods: We compared and contrasted a number of common scores (raw and relative gain scores) with our new method on two datasets, one simulated and the other empirical from a previous intervention study ( n = 180) employing a pre‐/post‐test design. Results: The outcomes of the common scores were clearly different, demonstrating a significant dependency on pre‐test scores. Only the new metric revealed a linear relationship to the knowledge baseline, was less skewed on the upper or lower extremes, and proved well suited to allow the calculation of negative learning gains. Employing the empirical dataset, the new method also confirmed the interaction effect of teaching formats with specific subgroups of learner characteristics. Conclusion: This work introduces a new weighted metric enabling meaningful comparisons between interventions based on a linear transformation. This method will form the basis to intertwine the calculation of test performance closely with the outcome of learning as an important factor reflecting teaching quality and efficacy. Its regular use can improve the transparency of teaching activities and outcomes, contribute to forming rounded judgements of students' acquisition of knowledge and skills and enable valuable feedforward to develop and enhance curricular concepts. Abstract : Westphale et al. introduce a new metric for correcting pre‐test bias that enables student learning to be used as an indicator for teaching quality.