Utilizing Response Times in Computerized Classification Testing. (July 2015)

• Record Type: Journal Article
• Authors: Sie, Haskell; Finkelman, Matthew D.; Riley, Barth; Smits, Niels
• Is Part Of: Applied psychological measurement. Volume 39:Number 5(2015)
• Page Start: 389
• Page End: 405
• Publication Date: 2015-07
• Subjects: item response theory -- Computerized Classification Testing -- response time -- lognormal model -- Sequential Probability Ratio Test; Psychometrics -- Periodicals; Psychological tests -- Periodicals; Psychology, Applied -- Periodicals; 150.1519505
• DOI: 10.1177/0146621615569504
• Languages: English
• ISSNs: 0146-6216

Abstract

A well-known approach in computerized mastery testing is to combine the Sequential Probability Ratio Test (SPRT) stopping rule with item selection to maximize Fisher information at the mastery threshold. This article proposes a new approach in which a time limit is defined for the test and examinees' response times are considered in both item selection and test termination. Item selection is performed by maximizing Fisher information per time unit, rather than Fisher information itself. The test is terminated once the SPRT makes a classification decision, the time limit is exceeded, or there is no remaining item that has a high enough probability of being answered before the time limit. In a simulation study, the new procedure showed a substantial reduction in average testing time while slightly improving classification accuracy compared with the original method. In addition, the new procedure reduced the percentage of examinees who exceeded the time limit.