Observing instructional quality in the context of school evaluation. (December 2016)

• Record Type: Journal Article
• Authors: Taut, Sandy; Rakoczy, Katrin
• Is Part Of: Learning and instruction. Volume 46(2016:Dec.)
• Page Start: 45
• Page End: 60
• Publication Date: 2016-12
• Subjects: Instructional quality -- Classroom observation -- School inspection -- External evaluation -- Generalizability; Learning -- Periodicals; Teaching -- Periodicals; Apprentissage -- Périodiques; Enseignement -- Périodiques; Learning; Teaching; Periodicals; Electronic journals; 370.1
• DOI: 10.1016/j.learninstruc.2016.08.003
• Languages: English
• ISSNs: 0959-4752

Abstract

Abstract: Observational measures of instructional quality are used in both research and evaluation contexts. In European school evaluation systems, one main intended use of classroom observation results is to inform schools' instructional improvement efforts. This study examined the quality of such diagnostic information in the context of a school evaluation system in Germany. The factor analytic results indicate that the empirical structure of the observation instrument lacks correspondence with its original normative model but does mirror a five-factor model based on recent literature, including classroom management, two different aspects of student orientation, cognitive activation, and classroom assessment. The generalizability analyses show that both indicators and observers represent important sources of measurement error. Additionally, their magnitude is larger for shorter observations (lesson segments). At least two, sometimes three, observers and between five and ten indicators per dimension are needed to achieve sufficiently reliable results, but results importantly depend on which dimension of instructional quality is being assessed. Highlights: Validity of observations in a school evaluation system examined by factor analysis. Reliability of the observations examined by generalizability studies. Empirical factor structure corresponded with models of instructional quality. Indicators and raters are relevant sources of error, esp. in shorter observations. Generalizability depends on the dimension of instructional quality assessed.