The Importance of Thinking Multivariately When Setting Subscale Cutoff Scores. (June 2022)

• Record Type: Journal Article
• Authors: Kroc, Edward; Olvera Astivia, Oscar L.
• Is Part Of: Educational and psychological measurement. Volume 82:Number 3(2022)
• Page Start: 517
• Page End: 538
• Publication Date: 2022-06
• Subjects: measurement -- cutoff scores -- multivariate analysis -- reliability -- classification; Educational tests and measurements -- Periodicals; Psychological tests -- Periodicals; 151.205
• DOI: 10.1177/00131644211023569
• Languages: English
• ISSNs: 0013-1644

Abstract

Setting cutoff scores is one of the most common practices when using scales to aid in classification purposes. This process is usually done univariately where each optimal cutoff value is decided sequentially, subscale by subscale. While it is widely known that this process necessarily reduces the probability of "passing" such a test, what is not properly recognized is that such a test loses power to meaningfully discriminate between target groups with each new subscale that is introduced. We quantify and describe this property via an analytical exposition highlighting the counterintuitive geometry implied by marginal threshold-setting in multiple dimensions. Recommendations are presented that encourage applied researchers to think jointly, rather than marginally, when setting cutoff scores to ensure an informative test.