A prediction-based method to estimate student learning outcome: Impact of response rate and gender differences on evaluation results. (4th May 2021)

• Record Type: Journal Article
• Authors: Grebener, Binia-Laureen; Barth, Janina; Anders, Sven; Beißbarth, Tim; Raupach, Tobias
• Is Part Of: Medical teacher. Volume 43:Number 5(2021)
• Page Start: 524
• Page End: 530
• Publication Date: 2021-05-04
• Subjects: Evaluation -- learning outcome -- self-assessment -- predictions -- gender; Medical education -- Periodicals; 610.711
• DOI: 10.1080/0142159X.2020.1867714
• Languages: English
• ISSNs: 0142-159X

Abstract

Abstract: Background: Low response rates threaten the reliability and validity of student evaluations of teaching. Previous research has shown that asking students to predict how satisfied their fellow students were with a course produces reliable results at lower response rates. The aim of this study was to investigate whether this prediction-based method can also be used to evaluate student learning outcome. Methods: Before and after a cardiorespiratory module, 128 fourth-year medical students provided self-assessments and predictions of performance on 27 specific learning objectives and took formative tests on the respective contents. Pre-post performance gain was compared across all three modalities. Results: Formative exam results indicated a performance gain of 63.0%. Self-assessed and prediction-based performance gains were identical (67.8%) but both slightly overestimated actual performance gain. Irrespective of the method used, a response rate of 20% was sufficient to produce reliable results. Compared to male students, females greatly overestimated their peers' performance which led to inflated performance gain values. Conclusions: Student self-assessments and predictions are equally valid sources of learning outcome measures, and low response rates are sufficient to produce stable results. When using a prediction-based approach, a tendency to overestimate learning outcome in female students needs to be taken into account.