Developing a level-specific checklist for assessing EFL writing. (January 2021)

• Record Type: Journal Article
• Authors: Lukácsi, Zoltán
• Is Part Of: Language testing. Volume 38:Number 1(2021)
• Page Start: 86
• Page End: 105
• Publication Date: 2021-01
• Subjects: Binary decisions -- level testing -- rating on a checklist -- scoring validity -- writing proficiency assessment; Language and languages -- Ability testing -- Periodicals; Language and languages -- Examinations -- Periodicals; 407.6
• DOI: 10.1177/0265532220916703
• Languages: English
• ISSNs: 0265-5322

Abstract

In second language writing assessment, rating scales and scores from human-mediated assessment have been criticized for a number of shortcomings including problems with adequacy, relevance, and reliability (Hamp-Lyons, 1990; McNamara, 1996; Weigle, 2002). In its testing practice, Euroexam International also detected that the rating scales for writing at B2 had limited discriminating power and did not adequately reflect finer shades of candidate ability. This study sought to investigate whether a level-specific checklist of binary choice items could be designed to yield results that accurately reflect differential degrees of ability in EFL essay writing at level B2. The participants were four language teachers working as independent raters. The study involved the task materials, operational rating scales, reported scores, and candidate scripts from the May 2017 test administration. In a mixed-methods strategy of inquiry, qualitative data from stimulated recall, think-aloud protocols, and semi-structured interviews informed statistical test and item analyses. The results indicated that the checklist items were more transparent, led to increased variance, and contributed to a more coherent candidate language profile than scores from the rating scales. The implications support the recommendation that checklists should be used for level-specific language proficiency testing (Council of Europe, 2001, p. 189).