Predicting communicative effectiveness in the international workplace: Support for TOEIC® Speaking test scores from linguistic laypersons. (April 2021)

• Record Type: Journal Article
• Authors: Schmidgall, Jonathan; Powers, Donald E.
• Is Part Of: Language testing. Volume 38:Number 2(2021)
• Page Start: 302
• Page End: 325
• Publication Date: 2021-04
• Subjects: Communicative effectiveness -- comprehensibility -- English as a lingua franca -- speaking -- validation -- workplace; Language and languages -- Ability testing -- Periodicals; Language and languages -- Examinations -- Periodicals; 407.6
• DOI: 10.1177/0265532220941803
• Languages: English
• ISSNs: 0265-5322

Abstract

In this study we examined the extent to which TOEIC® Speaking test scores relate to evaluations by professionals in the international workplace, the target language use domain of TOEIC tests. Linguistic laypersons in 10 countries were invited to participate in an online research survey. The survey incorporated a stratified sample of test-taker ( N = 99) responses to three representative tasks from the TOEIC Speaking test (reading a text aloud, responding to questions, expressing an opinion) that were cast as workplace role-play tasks. After completing each role-play task, participants used brief, descriptive six-point rating scales to rate the communicative effectiveness (comprehensibility, task fulfillment, elaboration, and coherence) of each of several speakers. Communicative effectiveness ratings from linguistic laypersons were strongly correlated with TOEIC Speaking test scaled scores ( r = 0.84). In addition, regression analysis was used to plot the relationship between layperson and test-based evaluations of speaking proficiency. Results suggested that test takers' performances can be expected to be perceived as effective at score ranges typically associated with important decisions. The results are discussed in terms of their implications for claims about the generalizability of TOEIC Speaking test score interpretations in relation to the evaluations of linguistic laypersons in the international workplace.