Validity Evidence Supporting Use of Anchoring Vignettes to Measure Teaching Practice. Issue 3 (3rd July 2019)

• Record Type: Journal Article
• Authors: Kaufman, Julia H.; Engberg, John; Hamilton, Laura S.; Yuan, Kun; Hill, Heather C.
• Is Part Of: Educational assessment. Volume 24:Issue 3(2019)
• Page Start: 155
• Page End: 188
• Publication Date: 2019-07-03
• Subjects: Educational tests and measurements -- United States -- Periodicals; 371.26097305
• DOI: 10.1080/10627197.2019.1615374
• Languages: English
• ISSNs: 1062-7197

Abstract

ABSTRACT: High-quality measures of instructional practice are essential for research and evaluation of innovative instructional policies and programs. However, existing measures have generally proven inadequate because of cost and validity issues. This paper addresses two potential drawbacks of survey self-report measures: variation in teachers' interpretation of response scales and their interpretation of survey questions. To address these drawbacks, researchers tested out use of "anchoring vignettes" in teacher surveys to capture information about teaching practice, and they gathered validity evidence in regard to their use as a tool for adjusting teachers' survey self-reports about their instructional practices for research purposes, or potentially to inform professional development. Data from 65 teachers in grades 4-9 responding to our survey suggested that vignette adjustments were reliable and valid for some instructional practices more than others. For some instructional practices, researchers found significant and high correlations between teachers' adjusted survey self-rating, through use of anchoring vignettes, and previous observation ratings of teachers' instruction, including ratings from several widely-used observation rubrics. These results suggest that anchoring vignettes may provide an efficient, cost-effective method for gathering data on teachers' instruction.