Embedded Standard Setting: Aligning Standard‐Setting Methodology with Contemporary Assessment Design Principles. Issue 1 (26th February 2020)

• Record Type: Journal Article
• Authors: Lewis, Daniel; Cook, Robert
• Is Part Of: Educational measurement. Volume 39:Issue 1(2020)
• Page Start: 8
• Page End: 21
• Publication Date: 2020-02-26
• Subjects: evidence‐centered design -- embedded standard setting -- item alignment -- principled assessment design -- standard setting; Educational tests and measurements -- Periodicals; Educational tests and measurements -- United States -- Periodicals; 371.262
• DOI: 10.1111/emip.12318
• Languages: English
• ISSNs: 0731-1745

Abstract

Abstract: In this paper we assert that the practice of principled assessment design renders traditional standard‐setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance standards that augments and is embedded within existing principled assessment design practices. Under ESS, achievement level descriptor writing and the targeting of items to specified achievement levels are viewed as the fundamental standard‐setting activities and cut scores are estimated by optimizing the relationship between the item targets and the empirically based grouping of items. If the relationship between item targets and empirical data are sufficiently strong, then the traditional standard‐setting workshop is either unnecessary or altered in nature and significantly reduced in scope. We provide empirical evidence supporting the efficacy of ESS and describe how ESS is embedded throughout the assessment lifecycle.