Relation between examinees' true knowledge and examination scores: systematic review and exemplary calculations on Pick-N items. (November 2022)

• Record Type: Journal Article
• Authors: Schmidt, Dennis; Raupach, Tobias; Wiegand, Annette; Herrmann, Manfred; Kanzow, Philipp
• Is Part Of: Educational research review. Volume 37(2022)
• Publication Date: 2022-11
• Subjects: k from n -- Multiple correct -- Multiple response -- Multiple-answer -- Multiple-mark -- Multiple-select -- n out of many; Education -- Research -- Periodicals; 370.7205
• DOI: 10.1016/j.edurev.2022.100483
• Languages: English
• ISSNs: 1747-938X

Abstract

Abstract: This manuscript focusing on Pick-N items is the second of two manuscripts regarding scoring approaches of two specific multiple-select item types commonly used to assess knowledge in written examinations. In contrast to other multiple-select item types, the number of true answer options to be marked within each Pick-N item is disclosed to examinees. As various scoring methods for Pick-N items exist, the present study aimed to help educators make informed choices about the use of Pick-N items, the scoring method to be selected, and related aspects (i.e. defining appropriate examination pass marks). Available scoring methods for conventional multiple-select items and Pick-N items were systematically identified from the literature. Their statistical parameters were compared by assessing the metrics available information included and expected chance scores from random guessing. The study further aimed to examine the relation between examinees' true knowledge and expected scoring results when using Pick-N items. Highlights: Pick-N items are one specific type of multiple-select items. Scoring of Pick-N items is complex. 24 scoring methods have been identified from literature. Statistical parameters were calculated for each scoring method. Relation between examinees' true knowledge and expected scoring results was assessed.