Investigating Robustness of Item Response Theory Proficiency Estimators to Atypical Response Behaviors Under Two‐Stage Multistage Testing. Issue 2 (15th June 2016)

• Record Type: Journal Article
• Authors: Kim, Sooyeon; Moses, Tim
• Is Part Of: ETS research report series. Issue 2(2016:Dec.)
• Page Start: 1
• Page End: 23
• Publication Date: 2016-06-15
• Subjects: Multistage testing -- proficiency estimator -- atypical response -- item response theory -- Bayesian estimator -- GRE® General Test; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/ets2.12111
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract: The purpose of this study is to evaluate the extent to which item response theory (IRT) proficiency estimation methods are robust to the presence of aberrant responses under the GRE® General Test multistage adaptive testing (MST) design. To that end, a wide range of atypical response behaviors affecting as much as 10% of the test items were simulated using a generic GRE 2‐stage MST and the 2‐parameter logistic (2PL) IRT model. As expected, some differences were found among the 5 estimators in terms of the recovery of the true theta ability; for example, Bayesian estimators had lower error variance and their estimates were regressed to the mean. Once the IRT theta estimates were scaled onto a comparable reporting score scale, however, it was found that all the estimation methods investigated, including the one currently used to score GRE MSTs, were equally robust under the simulated conditions. Abstract : Report Number: ETS RR‐16–22