Interpreting Probabilistic Classifications From Diagnostic Psychometric Models. Issue 2 (21st March 2019)

• Record Type: Journal Article
• Authors: Bradshaw, Laine; Levy, Roy
• Is Part Of: Educational measurement. Volume 38:Issue 2(2019)
• Page Start: 79
• Page End: 88
• Publication Date: 2019-03-21
• Subjects: cognitive diagnosis models -- diagnostic classification models -- diagnostic model reporting -- probabilistic classification -- score interpretation; Educational tests and measurements -- Periodicals; Educational tests and measurements -- United States -- Periodicals; 371.262
• DOI: 10.1111/emip.12247
• Languages: English
• ISSNs: 0731-1745

Abstract

Abstract: Although much research has been conducted on the psychometric properties of cognitive diagnostic models, they are only recently being used in operational settings to provide results to examinees and other stakeholders. Using this newer class of models in practice comes with a fresh challenge for diagnostic assessment developers: effectively reporting results and supporting end users to accurately interpret results. Achieving the goal of communicating results in a way that leads users of the assessment to make accurate interpretations requires a prerequisite step that cannot be taken for granted. The assessment developers must first accurately interpret results from a psychometric, or measurement, standpoint. Through this article, we seek to begin a discussion about reasonable interpretations of the results that classification‐based models provide about examinees. Interpretations from published research and ongoing practice show different—and sometimes conflicting—ways to interpret these results. This article seeks to formalize a comparison, critique, and discussion among the interpretations. Before beginning this discussion, we first present background on the results provided by classification‐based models regarding the examinees. We then structure our discussion around key questions an assessment development team needs to answer themselves prior to constructing reports and interpretative guides for end users of the assessment.