Adjusting for Ability Differences of Equating Samples When Randomization Is Suboptimal. Issue 3 (2nd April 2022)

• Record Type: Journal Article
• Authors: Kim, Sooyeon; Walker, Michael E.
• Is Part Of: Educational measurement. Volume 41:Issue 3(2022)
• Page Start: 26
• Page End: 37
• Publication Date: 2022-04-02
• Subjects: equating -- minimum discriminant information adjustment -- random groups design; Educational tests and measurements -- Periodicals; Educational tests and measurements -- United States -- Periodicals; 371.262
• DOI: 10.1111/emip.12506
• Languages: English
• ISSNs: 0731-1745

Abstract

Abstract: Test equating requires collecting data to link the scores from different forms of a test. Problems arise when equating samples are not equivalent and the test forms to be linked share no common items by which to measure or adjust for the group nonequivalence. Using data from five operational test forms, we created five pairs of research forms for each form, such that the equating relationship between each pair was known. Then we compared five approaches to adjusting for group nonequivalence in a situation where not only was group equivalence questionable, but the number of common items was small. We used a resampling approach to evaluate the linking accuracy of group adjustment using sample weights via minimum discriminant information adjustment (MDIA) using test takers' collateral (demographic) information, a weak anchor of only three items, or a mix of both. Overall, the use of both sample weights via MDIA and a weak anchor produced the most accurate result, while the direct (random groups) linking method assuming group equivalence produced the least accurate result due to nontrivial bias. For all five research forms, using both collateral information and anchor items only marginally improved linking accuracy compared to using the weak anchor alone.