Investigating the Practical Consequences of Model Misfit in Unidimensional IRT Models. (September 2017)

• Record Type: Journal Article
• Authors: Crişan, Daniela R.; Tendeiro, Jorge N.; Meijer, Rob R.
• Is Part Of: Applied psychological measurement. Volume 41:Number 6(2017)
• Page Start: 439
• Page End: 455
• Publication Date: 2017-09
• Subjects: item response theory -- model fit -- consequences of model violation; Psychometrics -- Periodicals; Psychological tests -- Periodicals; Psychology, Applied -- Periodicals; 150.1519505
• DOI: 10.1177/0146621617695522
• Languages: English
• ISSNs: 0146-6216

Abstract

In this article, the practical consequences of violations of unidimensionality on selection decisions in the framework of unidimensional item response theory (IRT) models are investigated based on simulated data. The factors manipulated include the severity of violations, the proportion of misfitting items, and test length. The outcomes that were considered are the precision and accuracy of the estimated model parameters, the correlations of estimated ability (θ ^ ) and number-correct (NC ) scores with the true ability (θ ), the ranks of the examinees and the overlap between sets of examinees selected based on eitherθ, θ ^, orNC scores, and the bias in criterion-related validity estimates. Results show that theθ ^ values were unbiased by violations of unidimensionality, but their precision decreased as multidimensionality and the proportion of misfitting items increased; the estimated item parameters were robust to violations of unidimensionality. The correlations betweenθ, θ ^, andNC scores, the agreement between the three selection criteria, and the accuracy of criterion-related validity estimates are all negatively affected, to some extent, by increasing levels of multidimensionality and the proportion of misfitting items. However, removing the misfitting items only improved the results in the case of severe multidimensionality and large proportion of misfitting items, and deteriorated them otherwise.