A Latent Class Signal Detection Model for Rater Scoring with Ordered Perceptual Distributions. Issue 1 (18th February 2020)

• Record Type: Journal Article
• Authors: DeCarlo, Lawrence T.; Zhou, Xiaoliang
• Is Part Of: Journal of educational measurement. Volume 58:Issue 1(2021)
• Page Start: 31
• Page End: 53
• Publication Date: 2020-02-18
• Subjects: Educational tests and measurements -- Periodicals; 371.2605
• DOI: 10.1111/jedm.12265
• Languages: English
• ISSNs: 0022-0655

Abstract

Abstract: In signal detection rater models for constructed response (CR) scoring, it is assumed that raters discriminate equally well between different latent classes defined by the scoring rubric. An extended model that relaxes this assumption is introduced; the model recognizes that a rater may not discriminate equally well between some of the scoring classes. The extension recognizes a different type of rater effect and is shown to offer useful tests and diagnostic plots of the equal discrimination assumption, along with ways to assess rater accuracy and various rater effects. The approach is illustrated with an application to a large‐scale language test.