Two Statistical Tests for the Detection of Item Compromise. (August 2022)

• Record Type: Journal Article
• Authors: van der Linden, Wim J.
• Is Part Of: Journal of educational and behavioral statistics. Volume 47:Number 4(2022)
• Page Start: 485
• Page End: 504
• Publication Date: 2022-08
• Subjects: continuous testing -- fixed-form testing -- item compromise -- item response theory -- lognormal response-time model -- statistical hypothesis testing; Educational statistics -- Periodicals; Social sciences -- Statistical methods -- Periodicals; 370.2
• DOI: 10.3102/10769986221094789
• Languages: English
• ISSNs: 1076-9986

Abstract

Two independent statistical tests of item compromise are presented, one based on the test takers' responses and the other on their response times (RTs) on the same items. The tests can be used to monitor an item in real time during online continuous testing but are also applicable as part of post hoc forensic analysis. The two test statistics are simple intuitive quantities as the sum of the responses and RTs observed for the test takers on the item. Common features of the tests are ease of interpretation and computational simplicity. Both tests are uniformly most powerful under the assumption of known ability and speed parameters for the test takers. Examples of power functions for items with realistic parameter values suggest maximum power for 20–30 test takers with item preknowledge for the response-based test and 10–20 test takers for the RT-based test.