"Digging for Gold" or "Sticking to the Criteria": Teachers' Rationales When Serving as Professional Raters. Issue 2 (23rd February 2020)

• Record Type: Journal Article
• Authors: Jølle, Lennart; Skar, Gustaf B.
• Is Part Of: Scandinavian journal of educational research. Volume 64:Issue 2(2020)
• Page Start: 283
• Page End: 299
• Publication Date: 2020-02-23
• Subjects: Writing assessment -- reliability -- validity -- rater training; Education -- Research -- Periodicals; 370.72
• DOI: 10.1080/00313831.2018.1541821
• Languages: English
• ISSNs: 0031-3831

Abstract

ABSTRACT: This paper reports findings from a project called "The National Panel of Raters" (NPR) that took place within a writing test programme in Norway (2010–2016). A recent research project found individual differences between the raters in the NPR. This paper reports results from an explorative follow up-study where 63 NPR members were surveyed with 23 items that were dilemma-like in the sense that deviating from the NPR rules would follow another—but socially acceptable—rationale. Four NPR members participated in a follow-up interview in which they motivated why they had agreed or disagreed with certain items. The results indicate two distinctly different stances toward rating work, with one stance threatening the validity of the scoring process.