Developing an automated writing placement system for ESL learners. Issue 3 (3rd July 2018)

• Record Type: Journal Article
• Authors: Yannakoudakis, Helen; Andersen, Øistein E; Geranpayeh, Ardeshir; Briscoe, Ted; Nicholls, Diane
• Is Part Of: Applied measurement in education. Volume 31:Issue 3(2018)
• Page Start: 251
• Page End: 267
• Publication Date: 2018-07-03
• Subjects: Educational tests and measurements -- United States -- Periodicals; Educational tests and measurements -- Periodicals; Education -- Standards -- Periodicals; 371.26
• DOI: 10.1080/08957347.2018.1464447
• Languages: English
• ISSNs: 0895-7347

Abstract

ABSTRACT: There are quite a few challenges in the development of an automated writing placement model for non-native English learners, among them the fact that exams that encompass the full range of language proficiency exhibited at different stages of learning are hard to design. However, acquisition of appropriate training data that are relevant to the task at hand is essential in the development of the model. Using the Cambridge Learner Corpus writing scores, which have been subsequently benchmarked to Common European Framework of Reference for Languages (CEFR) levels, we conceptualize the task as a supervised machine learning problem, and primarily focus on developing a generic writing model. Such an approach facilitates the modeling of truly consistent, internal marking criteria regardless of the prompt delivered, which has the additional advantage of requiring smaller dataset sizes and not necessarily requiring re-training or tuning for new tasks. The system is developed to predict someone's proficiency level on the CEFR scale, which allows learners to point to a specific standard of achievement. We furthermore integrate our model into Cambridge English Write & Improve TM —a freely available, cloud-based tool that automatically provides diagnostic feedback to non-native English language learners at different levels of granularity—and examine its use.