Weighting Test Samples in IRT Linking and Equating: Toward an Improved Sampling Design for Complex Equating. Issue 2 (December 2013)

• Record Type: Journal Article
• Authors: Qian, Jiahe; Jiang, Yanming; von Davier, Alina A.
• Is Part Of: ETS research report series. Issue 2(2013:Dec.)
• Page Start: i
• Page End: 31
• Publication Date: 2013-12
• Subjects: sampling design -- target population -- subsample -- poststratification -- raking -- complete grouped jackknifing; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/j.2333-8504.2013.tb02346.x
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract: Several factors could cause variability in item response theory (IRT) linking and equating procedures, such as the variability across examinee samples and/or test items, seasonality, regional differences, native language diversity, gender, and other demographic variables. Hence, the following question arises: Is it possible to select optimal samples of examinees so that the IRT linking and equating can be more precise at an administration level as well as over a large number of administrations? This is a question of optimal sampling design in linking and equating. To obtain an improved sampling design for invariant linking and equating across testing administrations, we applied weighting techniques to yield a weighted sample distribution that is consistent with the target population distribution. The goal is to obtain a stable Stocking‐Lord test characteristic curve (TCC) linking and a true‐score equating that is invariant across administrations. To study the weighting effects on linking, we first selected multiple subsamples from a data set. We then compared the linking parameters from subsamples with those from the data and examined whether the linking parameters from the weighted sample yielded smaller mean square errors (MSE) than those from the unweighted subsample. To study the weighting effects on true‐score equating, we also compared the distributions of the equated scores. Generally, the findings were that the weighting produced good results.