Artificial intelligence in educational assessment: 'Breakthrough? Or buncombe and ballyhoo?'. (5th July 2021)

• Record Type: Journal Article
• Authors: Gardner, John; O'Leary, Michael; Yuan, Li
• Is Part Of: Journal of computer assisted learning. Volume 37:Number 5(2021)
• Page Start: 1207
• Page End: 1216
• Publication Date: 2021-07-05
• Subjects: artificial intelligence -- automated essay scoring -- big data -- computerized adaptive tests -- learning analytics -- machine learning; Computer-assisted instruction -- Periodicals; 371.334
• DOI: 10.1111/jcal.12577
• Languages: English
• ISSNs: 0266-4909

Abstract

Abstract: Artificial Intelligence is at the heart of modern society with computers now capable of making process decisions in many spheres of human activity. In education, there has been intensive growth in systems that make formal and informal learning an anytime, anywhere activity for billions of people through online open educational resources and massive online open courses. Moreover, new developments in Artificial Intelligence‐related educational assessment are attracting increasing interest as means of improving assessment efficacy and validity, with much attention focusing on the analysis of the large volumes of process data being captured from digital assessment contexts. In evaluating the state of play of Artificial Intelligence in formative and summative educational assessment, this paper offers a critical perspective on the two core applications: automated essay scoring systems and computerized adaptive tests, along with the Big Data analysis approaches to machine learning that underpin them. Lay Description: What is already known about this topic?: AI and machine learning are established in summative assessments (SA) of learning The main types are Automated Essay Scoring (AES) and Computerised Adaptive Testing (CAT) Machine learning, based on Big Data and Learning Analytics, offers exciting possibilities in formative assessment (FA) What this paper adds?: A review of current AI and machine learning platforms in FA and SA Insights into the structure and design of CATs and AESs A review of current research on usage of Big Data/Learning Analytics for FA and SA A critique of the future for AI and machine learning in assessing higher order learning Implications for practice and/or policy: The use of AI for the assessment of higher order learning is not yet a feasible reality Sophisticated learning analytics can now offer closer alignment of AI to human judgment 'Trained' machine feedback can support self‐regulated learning in online environments Indicators are emerging from process data analysis on a learner's behaviour and affective state