A Simulation Study on the Performance of Different Reliability Estimation Methods. (December 2021)

• Record Type: Journal Article
• Authors: Edwards, Ashley A.; Joyner, Keanan J.; Schatschneider, Christopher
• Is Part Of: Educational and psychological measurement. Volume 81:Number 6(2021)
• Page Start: 1089
• Page End: 1117
• Publication Date: 2021-12
• Subjects: reliability -- simulation -- internal consistency; Educational tests and measurements -- Periodicals; Psychological tests -- Periodicals; 151.205
• DOI: 10.1177/0013164421994184
• Languages: English
• ISSNs: 0013-1644

Abstract

The accuracy of certain internal consistency estimators have been questioned in recent years. The present study tests the accuracy of six reliability estimators (Cronbach's alpha, omega, omega hierarchical, Revelle's omega, and greatest lower bound) in 140 simulated conditions of unidimensional continuous data with uncorrelated errors with varying sample sizes, number of items, population reliabilities, and factor loadings. Estimators that have been proposed to replace alpha were compared with the performance of alpha as well as to each other. Estimates of reliability were shown to be affected by sample size, degree of violation of tau equivalence, population reliability, and number of items in a scale. Under the conditions simulated here, estimates quantified by alpha and omega yielded the most accurate reflection of population reliability values. A follow-up regression comparing alpha and omega revealed alpha to be more sensitive to degree of violation of tau equivalence, whereas omega was affected greater by sample size and number of items, especially when population reliability was low.