Grouping Effects on Jackknifed Variance Estimation for Item Response Theory Scaling and Equating With Cluster‐Based Assessment Data. Issue 1 (27th April 2018)

• Record Type: Journal Article
• Authors: Wang, Lin; Qian, Jiahe; Lee, Yi‐Hsuan
• Is Part Of: ETS research report series. Issue 1(2018)
• Page Start: 1
• Page End: 14
• Publication Date: 2018-04-27
• Subjects: Grouped jackknife procedure -- random grouping method -- cluster‐based grouping method -- IRT‐based item calibration -- Stocking–Lord test characteristic curve linking; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/ets2.12204
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract: Educational assessment data are often collected from a set of test centers across various geographic regions, and therefore the data samples contain clusters. Such cluster‐based data may result in clustering effects in variance estimation. However, in many grouped jackknife variance estimation applications, jackknife groups are often formed by a random grouping method that ignores the cluster structures of the data. In this study, we constructed both random and cluster‐based jackknife groups for data known to have cluster structures and compared the jackknifed standard errors, yielded by two different grouping methods, of item response theory (IRT) scaling coefficient estimates and equated scores. Three independent data samples from an international test of English were used for the study. The cluster‐based jackknife group results showed relatively larger jackknifed standard errors of scaling coefficient estimates and scale scores than the results of the random jackknife groups for all three data samples. For cluster‐based assessment data, the cluster‐based jackknife approach provides a more appropriate way to estimate the standard errors of the parameters of IRT calibration, scaling, and equating analyses. Abstract : Report Number: ETS RR–18‐16