A Bayesian Random Block Item Response Theory Model for Forced-Choice Formats. (June 2020)

• Record Type: Journal Article
• Authors: Lee, HyeSun; Smith, Weldon Z.
• Is Part Of: Educational and psychological measurement. Volume 80:Number 3(2020)
• Page Start: 578
• Page End: 603
• Publication Date: 2020-06
• Subjects: forced-choice format -- Bayesian estimation -- random effect -- multidimensional item response theory (IRT); Educational tests and measurements -- Periodicals; Psychological tests -- Periodicals; 151.205
• DOI: 10.1177/0013164419871659
• Languages: English
• ISSNs: 0013-1644

Abstract

Based on the framework of testlet models, the current study suggests the Bayesian random block item response theory (BRB IRT) model to fit forced-choice formats where an item block is composed of three or more items. To account for local dependence among items within a block, the BRB IRT model incorporated a random block effect into the response function and used a Markov Chain Monte Carlo procedure for simultaneous estimation of item and trait parameters. The simulation results demonstrated that the BRB IRT model performed well for the estimation of item and trait parameters and for screening those with relatively low scores on target traits. As found in the literature, the composition of item blocks was crucial for model performance; negatively keyed items were required for item blocks. The empirical application showed the performance of the BRB IRT model was equivalent to that of the Thurstonian IRT model. The potential advantage of the BRB IRT model as a base for more complex measurement models was also demonstrated by incorporating gender as a covariate into the BRB IRT model to explain response probabilities. Recommendations for the adoption of forced-choice formats were provided along with the discussion about using negatively keyed items.