A Comparison of Raw‐to‐Scale Conversion Consistency Between Single‐ and Multiple‐Linking Using a Nonequivalent Groups Anchor Test Design. Issue 2 (24th July 2014)

• Record Type: Journal Article
• Authors: Liu, Jinghua; Guo, Hongwen; Dorans, Neil J.
• Is Part Of: ETS research report series. Issue 2(2014:Dec.)
• Page Start: 1
• Page End: 11
• Publication Date: 2014-07-24
• Subjects: Scale drift -- raw‐to‐scale conversion consistency -- single linking -- multiple linking; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/ets2.12014
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract : Maintaining score interchangeability and scale consistency is crucial for any testing programs that administer multiple forms across years. The use of a multiple linking design, which involves equating a new form to multiple old forms and averaging the conversions, has been proposed to control scale drift. However, the use of multiple linking often conflicts with the need for minimizing old item/form exposure and the need for pretesting. This study tried to find a balance point where the needs for equating, item/form exposure and controlling, and pretesting can be satisfied. Three equating scenarios were examined using real data: equating to one old form, equating to two old forms, or equating to three old forms. The finding is that the equating based on one old form produced persistent score drift and also showed increased variability in score means and standard deviations over time. In contrast, equating back to two or three old forms produced much more stable conversions and had less variation. Overall, equating based on multiple linking designs shows the promise of producing more consistent results and preventing scale drift. We recommend that testing programs and practitioners consider the use of multiple linking whenever possible. Abstract : Report Number: ETS RR–14–13