Condensed Mastery Profile Method for Setting Standards for Diagnostic Assessment Systems. Issue 4 (12th July 2017)

• Record Type: Journal Article
• Authors: Clark, A. K.; Nash, B.; Karvonen, M.; Kingston, N.
• Is Part Of: Educational measurement. Volume 36:Issue 4(2017:Winter)
• Page Start: 5
• Page End: 15
• Publication Date: 2017-07-12
• Subjects: body of work -- diagnostic classification modeling -- standard setting -- student profiles; Educational tests and measurements -- Periodicals; Educational tests and measurements -- United States -- Periodicals; 371.262
• DOI: 10.1111/emip.12162
• Languages: English
• ISSNs: 0731-1745

Abstract

Abstract: The purpose of this study was to develop a standard‐setting method appropriate for use with a diagnostic assessment that produces profiles of student mastery rather than a single raw or scale score value. The condensed mastery profile method draws from established holistic standard‐setting methods to use rounds of range finding and pinpointing to specify cut points between performance levels. Panelists are convened to review profiles of mastery and specify cut points between performance levels based on the total number of skills mastered. Following panelist specification of cut points, a statistical method is implemented to smooth cut points over grades to decrease between‐grade variability. Procedural evidence, including convergence plots, standard errors of pinpointing ratings, and panelist feedback, suggest the condensed mastery profile method is a useful and technically sound approach for setting performance standards for diagnostic assessment systems.