A Bayesian approach to improving measurement precision over multiple test occasions. (October 2020)

• Record Type: Journal Article
• Authors: Van Moere, Alistair; Hanlon, Sean
• Other Names: Green Anthony guest-editor.; Van Moere Alistair guest-editor.
• Is Part Of: Language testing. Volume 37:Number 4(2020)
• Page Start: 482
• Page End: 502
• Publication Date: 2020-10
• Subjects: Bayesian -- Lexile -- longitudinal -- prior scores -- reading comprehension -- repeated test-taking -- score changes -- score fluctuations -- uncertainty; Language and languages -- Ability testing -- Periodicals; Language and languages -- Examinations -- Periodicals; 407.6
• DOI: 10.1177/0265532220934203
• Languages: English
• ISSNs: 0265-5322

Abstract

In language assessment and in educational measurement more broadly, there is a tendency to interpret scores from single-administration tests as accurate indicators of a latent trait (e.g., reading ability). Even in contexts where learners receive multiple formative assessments throughout the year, estimates of student ability are determined based on the most recent assessment. This paper demonstrates a technique that incorporates prior test scores with current scores for learners who re-test periodically, in order to arrive at an estimate closer to the learners' true score. Over 21, 000 learners from two separate studies (EFL and native speaker) were tested for reading proficiency between three and five times each, over a one- to two-year period, on a multiple-choice reading test which reported reading ability in Lexile® measures. Applying Bayes theorem, prior scores and the most recent test score were combined with uncertainty parameters (i.e., measurement error) to produce new estimates of student ability. This is advantageous as prior administration data is re-used rather than discarded. The approach is recommended in the context of periodic low-stakes tests designed to measure proficiency gains over time, as well as for high-stakes tests as an alternative to allowing candidates to cherry-pick their highest score for university applications.