Of Small Beauties and Large Beasts: The Quality of Distractors on Multiple-Choice Tests Is More Important Than Their Quantity. Issue 4 (2nd October 2017)

• Record Type: Journal Article
• Authors: Papenberg, Martin; Musch, Jochen
• Is Part Of: Applied measurement in education. Volume 30:Issue 4(2017)
• Page Start: 273
• Page End: 286
• Publication Date: 2017-10-02
• Subjects: Educational tests and measurements -- United States -- Periodicals; Educational tests and measurements -- Periodicals; Education -- Standards -- Periodicals; 371.26
• DOI: 10.1080/08957347.2017.1353987
• Languages: English
• ISSNs: 0895-7347

Abstract

ABSTRACT: In multiple-choice tests, the quality of distractors may be more important than their number. We therefore examined the joint influence of distractor quality and quantity on test functioning by providing a sample of 5, 793 participants with five parallel test sets consisting of items that differed in the number and quality of distractors. Surprisingly, we found that items in which only the one best distractor was presented together with the solution provided the strongest criterion-related evidence of the validity of test scores and thus allowed for the most valid conclusions on the general knowledge level of test takers. Items that included the best distractor produced more reliable test scores irrespective of option number. Increasing the number of options increased item difficulty, but did not increase internal consistency when testing time was controlled for.