Judgment accuracy in experienced versus student teachers: Assessing essays in English as a foreign language. (January 2021)

• Record Type: Journal Article
• Authors: Jansen, Thorben; Vögelin, Cristina; Machts, Nils; Keller, Stefan; Köller, Olaf; Möller, Jens
• Is Part Of: Teaching and teacher education. Volume 97(2021)
• Publication Date: 2021-01
• Subjects: Teacher judgment accuracy -- Foreign language writing -- Automated essay scoring; Teaching -- Periodicals; Teachers -- Training of -- Periodicals; Enseignement -- Périodiques; Enseignants -- Formation -- Périodiques; Electronic journals; 371.10205
• DOI: 10.1016/j.tate.2020.103216
• Languages: English
• ISSNs: 0742-051X

Abstract

Abstract: Teacher judgment accuracy (TJA) is a central part of diagnostic competence. Research has revealed heterogeneous results regarding teacher characteristics as determinants of TJA. In the present experimental study N = 36 experienced and N = 47 student teachers holistically and analytically assessed four student essays. We compared the judgments of teachers and student teachers to machine scores and expert ratings. Experienced teachers' judgments were more negative than student teachers', and both judgments were more negative than those made by experts, and the machine. The results remained stable, even after controlling for content knowledge. They are discussed in relation to the assumption of an expert-blind-spot. Highlights: Examining teacher judgment accuracy when assessing students' texts. Machine-generated scores and expert ratings served as accuracy criteria. Experienced teachers' judgments were more negative than those of student teachers. The judgments of both groups were more negative than expert judgments and machine ratings. Results remained stable after controlling for teachers' content knowledge.