Validating the Slovenian national alignment to CEFR: The case of the B2 reading comprehension examination in English. (October 2015)

• Record Type: Journal Article
• Authors: Ilc, Gašper; Stopar, Andrej
• Is Part Of: Language testing. Volume 32:Number 4(2015:Oct.)
• Page Start: 443
• Page End: 462
• Publication Date: 2015-10
• Subjects: Common European Framework of Reference -- cut scores -- external validation -- reading comprehension -- relating -- socio-cognitive model; Language and languages -- Ability testing -- Periodicals; Language and languages -- Examinations -- Periodicals; 407.6
• DOI: 10.1177/0265532214562098
• Languages: English
• ISSNs: 0265-5322

Abstract

The paper examines the results of the CEFR alignment project for the Slovenian national examinations in English. The authors aim to validate externally the standard-setting procedures by adopting a socio-cognitive model of validation (Khalifa & Weir, 2009; Weir, 2005) to analyse the scoring, context and cognitive validity of three reading subtests: the Slovenian B2 national examination and the international examinations FCE and CAE, aligned with B2 and C1 respectively. The relative comparability between the three subtests is determined by analysing the results of tests that have been administered to a group of 80 test-takers (expected CEFR level: B2). The placement of the test-takers also reveals to what extent the judgements of the Slovenian panellists about CEFR levels coincide with those reported for FCE and CAE. The study thus also explores whether the high degree of agreement between the judges on the alignment panel can be solely attributed to their adequate and precise understanding of CEFR descriptors – which is directly mirrored in their setting of the cut scores and relating the examination to relevant CEFR levels – or whether it can also be ascribed to their shared educational, national and cultural background. The answers to these questions are paramount because they reveal the descriptive adequacy of CEFR descriptors and because different interpretations of CEFR levels can significantly affect national testing policies and, consequently, language teaching and testing.