Detecting DIF in Multidimensional Forced Choice Measures Using the Thurstonian Item Response Theory Model. (October 2021)

• Record Type: Journal Article
• Authors: Lee, Philseok; Joo, Seang-Hwane; Stark, Stephen
• Is Part Of: Organizational research methods. Volume 24:Number 4(2021)
• Page Start: 739
• Page End: 771
• Publication Date: 2021-10
• Subjects: noncognitive testing -- multidimensional forced choice measures -- measurement invariance -- differential item functioning -- Thurstonian item response theory -- Monte Carlo simulation; Organization -- Research -- Methodology -- Periodicals; Organizational sociology -- Research -- Methodology -- Periodicals; Management -- Research -- Methodology -- Periodicals; 302.350721
• DOI: 10.1177/1094428120959822
• Languages: English
• ISSNs: 1094-4281

Abstract

Although modern item response theory (IRT) methods of test construction and scoring have overcome ipsativity problems historically associated with multidimensional forced choice (MFC) formats, there has been little research on MFC differential item functioning (DIF) detection, where item refers to a block, or group, of statements presented for an examinee's consideration. This research investigated DIF detection with three-alternative MFC items based on the Thurstonian IRT (TIRT) model, using omnibus Wald tests on loadings and thresholds. We examined constrained and free baseline model comparisons strategies with different types and magnitudes of DIF, latent trait correlations, sample sizes, and levels of impact in an extensive Monte Carlo study. Results indicated the free baseline strategy was highly effective in detecting DIF, with power approaching 1.0 in the large sample size and large magnitude of DIF conditions, and similar effectiveness in the impact and no-impact conditions. This research also included an empirical example to demonstrate the viability of the best performing method with real examinees and showed how a DIF and a DTF effect size measure can be used to assess the practical significance of MFC DIF findings.