Adding Objectivity to Standard Setting: Evaluating Consequence Using the Conscious and Subconscious Weight Methods. Issue 1 (26th February 2020)

• Record Type: Journal Article
• Authors: Leventhal, Brian C.; Grabovsky, Irina
• Is Part Of: Educational measurement. Volume 39:Issue 1(2020)
• Page Start: 30
• Page End: 36
• Publication Date: 2020-02-26
• Subjects: classification -- licensure testing -- standard setting; Educational tests and measurements -- Periodicals; Educational tests and measurements -- United States -- Periodicals; 371.262
• DOI: 10.1111/emip.12316
• Languages: English
• ISSNs: 0731-1745

Abstract

Abstract: Standard setting is arguably one of the most subjective techniques in test development and psychometrics. The decisions when scores are compared to standards, however, are arguably the most consequential outcomes of testing. Providing licensure to practice in a profession has high stake consequences for the public. Denying graduation or forcing remediation has high‐impact consequences for students. Unfortunately, tests that classify individuals are subjected to false positive and false negative misclassifications. When determining a standard, standard setting panelists implicitly consider the negative consequences of the decisions made from test use. We propose the conscious weight method and subconscious weight method to bring more objectivity to the standard setting process. To do this, these methods quantify the relative harm of the negative consequences of false positive and false negative misclassification.