Sensitivity of test items to teaching quality. (April 2019)

• Record Type: Journal Article
• Authors: Naumann, Alexander; Rieser, Svenja; Musow, Stephanie; Hochweber, Jan; Hartig, Johannes
• Is Part Of: Learning and instruction. Volume 60(2019)
• Page Start: 41
• Page End: 53
• Publication Date: 2019-04
• Subjects: Teaching quality -- Instructional sensitivity -- Educational effectiveness -- Validity -- Classroom assessment; Learning -- Periodicals; Teaching -- Periodicals; Apprentissage -- Périodiques; Enseignement -- Périodiques; Learning; Teaching; Periodicals; Electronic journals; 370.1
• DOI: 10.1016/j.learninstruc.2018.11.002
• Languages: English
• ISSNs: 0959-4752

Abstract

Abstract: Instructional sensitivity is the psychometric capacity of tests or single items of capturing effects of classroom instruction. Yet, current item sensitivity measures' relationship to (a) actual instruction and (b) overall test sensitivity is rather unclear. The present study aims at closing these gaps by investigating test and item sensitivity to teaching quality, reanalyzing data from a quasi-experimental intervention study in primary school science education (1026 students, 53 classes, Mage = 8.79 years, SDage = 0.49, 50% female). We examine (a) the correlation of item sensitivity measures and the potential for cognitive activation in class and (b) consequences for test score interpretation when assembling tests from items varying in their degree of sensitivity to cognitive activation. Our study (a) provides validity evidence that item sensitivity measures may be related to actual classroom instruction and (b) points out that inferences on teaching drawn from test scores may vary due to test composition. Highlights: Our study engages the question of the extent to which test items are capable of capturing effects of classroom instruction. We model item sensitivity measures based on change scores and baseline-adjusted measures. We provide validity evidence that our item sensitivity measures may be related to actual classroom instruction. Inferences on teaching effectiveness may vary due to test composition in terms of items' instructional sensitivity. Item selection solely based on instructional sensitivity measures presumably leads to invalid inferences on teaching.