Relation between examinees' true knowledge and examination scores: systematic review and exemplary calculations on Pick-N items. (November 2022) - Schmidt, Dennis; Raupach, Tobias; Wiegand, Annette; Herrmann, Manfred; Kanzow, Philipp
Abstract: This manuscript focusing on Pick-N items is the second of two manuscripts regarding scoring approaches of two specific multiple-select item types commonly used to assess knowledge in written examinations. In contrast to other multiple-select item types, the number of true answer options to be marked within each Pick-N item is disclosed to examinees. As various scoring methods for Pick-N items exist, the present study aimed to help educators make informed choices about the use of Pick-N items, the scoring method to be selected, and related aspects (i.e. defining appropriate examination pass marks). Available scoring methods for conventional multiple-select items and Pick-N items were systematically identified from the literature. Their statistical parameters were compared by assessing the metrics available information included and expected chance scores from random guessing. The study further aimed to examine the relation between examinees' true knowledge and expected scoring results when using Pick-N items. Highlights: Pick-N items are one specific type of multiple-select items. Scoring of Pick-N items is complex. 24 scoring methods have been identified from literature. Statistical parameters were calculated for each scoring method. Relation between examinees' true knowledge and expected scoring results was assessed.