Administrators' Uses of Teacher Observation Protocol in Different Rating Contexts. Issue 1 (27th April 2018)

• Record Type: Journal Article
• Authors: Qi, Yi; Bell, Courtney A.; Jones, Nathan D.; Lewis, Jennifer M.; Witherspoon, Margaret W.; Redash, Amanda
• Is Part Of: ETS research report series. Issue 1(2018)
• Page Start: 1
• Page End: 19
• Publication Date: 2018-04-27
• Subjects: Observation -- teacher quality -- measurement of teaching -- rater cognition; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/ets2.12205
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract: Teacher observations are being used for high‐stakes purposes in states across the country, and administrators often serve as raters in teacher evaluation systems. This paper examines how the cognitive aspects of administrators' use of an observation instrument, a modified version of Charlotte Danielson's Framework for Teaching, interact with the complex and dynamic rating contexts in applied settings. Findings suggest that administrators' rating strategies and rating approaches vary as the characteristics of the rating contexts differ. Even shortly after training (and more so as time passed), raters used reasoning strategies not supported by their training to make scoring decisions. We discuss the implications of the findings for the training of raters and the development of evaluation systems in high‐stakes contexts. Abstract : Report Number: ETS RR–18‐18