Parameter recovery of a cognitive diagnostic model: evidence from a simulation study. (2016)

• Record Type: Journal Article
• Authors: Galeshi, Roofia; Skaggs, Gary
• Is Part Of: International journal of quantitative research in education. Volume 3: Number 4(2016)
• Page Start: 223
• Page End: 241
• Publication Date: 2016
• Subjects: cognitive modelling -- cognitive diagnostic model -- CDM -- parameter recovery -- compensatory reparameterised unified model -- CRUM -- simulation -- classification accuracy -- research design; Education -- Research -- Methodology -- Periodicals; Training -- Research -- Periodicals; Quantitative research -- Periodicals; 370.721
• Languages: English
• ISSNs: 2049-5986

Abstract

Although item response theory is a valuable summative assessment tool, cognitive diagnostic models could potentially be a valuable instructional and formative assessment instrument. Using simulation methods, we have examined one of the existing additive models - the compensatory reparameterised unified model (CRUM) - for its parameter recovery and classification accuracy under various research designs. In order for models such as the CRUM to be useful, it is essential to establish their parameter recovery and classification accuracy. This study simulated data for sample sizes of 50, 100, 500, 1, 000, 5, 000, and 10, 000 examinees with various attribute-item combinations: 7 items with 3 attributes, 15 items with 4 attributes, 31 items with 5 attributes, and 35 items with 3 attributes to be evaluated for its parameter recovery. The results suggest that the accuracy of measuring examinees' attribute mastery depends heavily on the combination of the number of items and attributes as well as the sample size. Item parameter estimation was almost equally accurate for datasets of sample sizes greater than 100. The more complex items required larger sample sizes to maintain its consistency of parameter estimation.