A Regression Discontinuity Design Framework for Controlling Selection Bias in Evaluations of Differential Item Functioning. (December 2022)

• Record Type: Journal Article
• Authors: Koziol, Natalie A.; Goodrich, J. Marc; Yoon, HyeonJin
• Is Part Of: Educational and psychological measurement. Volume 82:Number 6(2022)
• Page Start: 1247
• Page End: 1277
• Publication Date: 2022-12
• Subjects: differential item functioning (DIF) -- logistic regression -- regression discontinuity design -- selection bias; Educational tests and measurements -- Periodicals; Psychological tests -- Periodicals; 151.205
• DOI: 10.1177/00131644211068440
• Languages: English
• ISSNs: 0013-1644

Abstract

Differential item functioning (DIF) is often used to examine validity evidence of alternate form test accommodations. Unfortunately, traditional approaches for evaluating DIF are prone to selection bias. This article proposes a novel DIF framework that capitalizes on regression discontinuity design analysis to control for selection bias. A simulation study was performed to compare the new framework with traditional logistic regression, with respect to Type I error and power rates of the uniform DIF test statistics and bias and root mean square error of the corresponding effect size estimators. The new framework better controlled the Type I error rate and demonstrated minimal bias but suffered from low power and lack of precision. Implications for practice are discussed.