An Investigation of the Impact of Misrouting Under Two‐Stage Multistage Testing: A Simulation Study. (18th April 2014)

• Record Type: Journal Article
• Authors: Kim, Sooyeon; Moses, Tim
• Is Part Of: ETS research report series. Volume 1(2014)
• Page Start: 1
• Page End: 13
• Publication Date: 2014-04-18
• Subjects: Multistage testing -- routing -- panel assembly -- adaptive test -- bias; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/ets2.12000
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract : The purpose of this study was to investigate the potential impact of misrouting under a 2‐stage multistage test (MST) design, which includes 1 routing and 3 second‐stage modules. Simulations were used to create a situation in which a large group of examinees took each of the 3 possible MST paths (high, middle, and low). We compared differences in examinees' scores associated with different paths under 2 MST assembly conditions: a small‐difference condition in which 3 modules at Stage 2 overlap in difficulty, and a large‐difference condition in which the 3 modules at Stage 2 are distinct in difficulty. We also compared examinees' score differences associated with the target MST path with the ones obtained when the second‐stage module was 1 level of difficulty above or below the target module. When the second‐stage modules overlapped in difficulty, the score differences (i.e., bias) associated with different MST paths were negligible for practical purposes. Similar trends appeared even when the second‐stage modules were significantly distinct in difficulty. The impact of misrouting was generally minimal under the MST design used in this study. Abstract : Report Number: ETS RR–14–01