The measurement of calibration in real contexts. (February 2017)

• Record Type: Journal Article
• Authors: Rutherford, Teomara
• Is Part Of: Learning and instruction. Volume 47(2017:Feb.)
• Page Start: 33
• Page End: 42
• Publication Date: 2017-02
• Subjects: Metacognitive monitoring -- Accuracy -- Calibration -- Data -- Self-regulated learning; Learning -- Periodicals; Teaching -- Periodicals; Apprentissage -- Périodiques; Enseignement -- Périodiques; Learning; Teaching; Periodicals; Electronic journals; 370.1
• DOI: 10.1016/j.learninstruc.2016.10.006
• Languages: English
• ISSNs: 0959-4752

Abstract

Abstract: Accurate judgment of performance, or calibration, is an important element of self-regulated learning (SRL) and itself has been an area of growing study. The current study contributes to work on calibration by presenting practical and predictive results of varying calibration measures from authentic educational data: elementary-aged students' interactions with a year-long digital mathematics curriculum. Comparison of predictive validity of measures show only small differences in explained variance in models predicting posttest performance while controlling for pretest. A combined model including Sensitivity and Specificity outperforms other single measures, confirming results in Schraw, Kuch, & Gutierrez (2013); however, results show that student patterns of calibration within these data differ from those assumed in simulation studies and these differences have implications for the calculability of popular calibration measures. Highlights: Calibration measures calculated from authentic educational data differ from simulated data. Patterns of missing data are problematic for calculation of common calibration measures. A model with confident and uncertain processes together explains the most variance in performance.