When Should Individual Ability Estimates Be Reported if Rapid Guessing Is Present?. Issue 3 (3rd July 2022)

• Record Type: Journal Article
• Authors: Rios, Joseph A.
• Is Part Of: Applied measurement in education. Volume 35:Issue 3(2022)
• Page Start: 222
• Page End: 236
• Publication Date: 2022-07-03
• Subjects: Educational tests and measurements -- United States -- Periodicals; Educational tests and measurements -- Periodicals; Education -- Standards -- Periodicals; 371.26
• DOI: 10.1080/08957347.2022.2103138
• Languages: English
• ISSNs: 0895-7347

Abstract

ABSTRACT: Testing programs are confronted with the decision of whether to report individual scores for examinees that have engaged in rapid guessing (RG). As noted by the Standards for Educational and Psychological Testing, this decision should be based on a documented criterion that determines score exclusion. To this end, a number of heuristic criteria (e.g., exclude all examinees with RG rates of 10%) have been adopted in the literature. Given that these criteria lack strong methodological support, the objective of this simulation study was to evaluate their appropriateness in terms of individual ability estimate and classification accuracy when manipulating both assessment and RG characteristics. The findings provide evidence that employing a common criterion for all examinees may be an ineffective strategy because a given RG percentage may have differing degrees of biasing effects based on test difficulty, examinee ability, and RG pattern. These results suggest that practitioners may benefit from establishing context-specific exclusion criteria that consider test purpose, score use, and targeted examinee trait levels.