Methods for Estimating Item-Score Reliability. Issue 7 (October 2018)

• Record Type: Journal Article
• Authors: Zijlmans, Eva A. O.; van der Ark, L. Andries; Tijmstra, Jesper; Sijtsma, Klaas
• Is Part Of: Applied psychological measurement. Volume 42:Issue 7(2018)
• Page Start: 553
• Page End: 570
• Publication Date: 2018-10
• Subjects: correction for attenuation -- Guttman's method λ6 -- item-score reliability -- latent class reliability coefficient -- method MS; Psychometrics -- Periodicals; Psychological tests -- Periodicals; Psychology, Applied -- Periodicals; 150.1519505
• DOI: 10.1177/0146621618758290
• Languages: English
• ISSNs: 0146-6216

Abstract

Reliability is usually estimated for a test score, but it can also be estimated for item scores. Item-score reliability can be useful to assess the item's contribution to the test score's reliability, for identifying unreliable scores in aberrant item-score patterns in person-fit analysis, and for selecting the most reliable item from a test to use as a single-item measure. Four methods were discussed for estimating item-score reliability: the Molenaar–Sijtsma method (method MS), Guttman's methodλ 6, the latent class reliability coefficient (method LCRC), and the correction for attenuation (method CA). A simulation study was used to compare the methods with respect to median bias, variability (interquartile range [IQR]), and percentage of outliers. The simulation study consisted of six conditions: standard, polytomous items, unequalα parameters, two-dimensional data, long test, and small sample size. Methods MS and CA were the most accurate. Method LCRC showed almost unbiased results, but large variability. Methodλ 6 consistently underestimated item-score reliabilty, but showed a smaller IQR than the other methods.