Comparability of students' writing performance on TOEFL iBT and in required university writing courses. (April 2019)

• Record Type: Journal Article
• Authors: Llosa, Lorena; Malone, Margaret E.
• Is Part Of: Language testing. Volume 36:Number 2(2019)
• Page Start: 235
• Page End: 263
• Publication Date: 2019-04
• Subjects: Extrapolation inference -- Independent task -- Integrated task -- TOEFL iBT -- validity -- writing assessment; Language and languages -- Ability testing -- Periodicals; Language and languages -- Examinations -- Periodicals; 407.6
• DOI: 10.1177/0265532218763456
• Languages: English
• ISSNs: 0265-5322

Abstract

Investigating the comparability of students' performance on TOEFL writing tasks and actual academic writing tasks is essential to provide backing for the extrapolation inference in the TOEFL validity argument (Chapelle, Enright, & Jamieson, 2008). This study compared 103 international non-native-English-speaking undergraduate students' performance on two TOEFL iBT ® writing tasks with their performance in required writing courses in US universities as measured by instructors' ratings of student proficiency, instructor-assigned grades on two course assignments, and five dimensions of writing quality of the first and final drafts of those course assignments: grammatical, cohesive, rhetorical, sociopragmatic, and content control. Also, the quality of the writing on the TOEFL writing tasks was compared with the first and final drafts of responses to written course assignments using a common analytic rubric along the five dimensions. Correlations of scores from TOEFL tasks (Independent, Integrated, and the total Writing section) with instructor ratings of students' overall English proficiency and writing proficiency were moderate and significant. However, only scores on the Integrated task and the Writing section were correlated with instructor-assigned grades on course assignments. Correlations between scores on TOEFL tasks and all dimensions of writing quality were positive and significant, though of lower magnitude for final drafts than for first drafts. The TOEFL scores were most highly correlated with cohesive and grammatical control and had the lowest correlations with rhetorical organization. The quality of the writing on the TOEFL tasks was comparable to that of the first drafts of course assignment but not the final drafts. These findings provide backing for the extrapolation inference, suggesting that the construct of academic writing proficiency as assessed by TOEFL "accounts for the quality of linguistic performance in English-medium institutions of higher education" (Chapelle, Enright, & Jamieson, 2008, p. 21).