At a first glance – How reliable and valid is the thin slices technique to assess instructional quality?. (August 2021)

• Record Type: Journal Article
• Authors: Begrich, Lukas; Kuger, Susanne; Klieme, Eckhard; Kunter, Mareike
• Is Part Of: Learning and instruction. Volume 74(2021)
• Publication Date: 2021-08
• Subjects: Instructional quality -- Assessment -- Thin slices ratings -- First impressions; Learning -- Periodicals; Teaching -- Periodicals; Apprentissage -- Périodiques; Enseignement -- Périodiques; Learning; Teaching; Periodicals; Electronic journals; 370.1
• DOI: 10.1016/j.learninstruc.2021.101466
• Languages: English
• ISSNs: 0959-4752

Abstract

Abstract: Various approaches of assessing instructional quality have emerged in educational research. In this article, we present two studies that apply the thin slices procedure, investigating the reliability and validity of the ratings of three dimensions of instructional quality based solely on the first impressions of untrained social observers. Thirty undergraduate students rated 30-s clips from English lessons (Study 1) and Math lessons (Study 2) regarding three quality dimensions. The findings suggest high reliability in these ratings. Multilevel confirmatory analyses suggested construct validity in terms of differentiation between the three dimensions of instructional quality. Finally, we found some overlap between the thin slices ratings of classroom management and constructive support with ratings of trained raters based on observations of full lessons, as well as students' ratings of these dimensions. We discuss these results with respect to the potential of first impressions of untrained observers to measure instructional quality. Highlights: Ratings of instructional quality based on first impressions of untrained observers. Very high consensus among observers shows high reliability of thin slices ratings. Observers can distinguish between different aspects of instructional quality. Ratings show some overlap with other indicators of instructions quality. Results are broadly comparable across two different school subjects.