Using Existing Data to Inform Development of New Item Types. Issue 1 (22nd January 2020)

• Record Type: Journal Article
• Authors: Guo, Hongwen; Ling, Guangming; Frankel, Lois
• Is Part Of: ETS research report series. Issue 1(2020)
• Page Start: 1
• Page End: 16
• Publication Date: 2020-01-22
• Subjects: Test reliability -- partial credit -- nonfunctioning option -- data‐driven simulation; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/ets2.12284
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract : With advances in technology, researchers and test developers are developing new item types to measure complex skills like problem solving and critical thinking. Analyzing such items is often challenging because of their complicated response patterns, and thus it is important to develop psychometric methods for practitioners and researchers to analyze these new item types. In this study, we describe a generic approach that involves data‐driven analyses and expert feedback from different research areas so that the analysis results can provide valuable information to test developers and researchers on how complex item types contribute to score reliability and validity and on how to make the test more efficient and reliable in measuring complex skills. A real data example was used to illustrate how to identify nonfunctioning options that might be removed from the test and whether partial credit for certain response selections can be considered. Abstract : Report Number: ETS RR‐20‐01