What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. (April 2021)

• Record Type: Journal Article
• Authors: Lim, Lisa-Angelique; Gentili, Sheridan; Pardo, Abelardo; Kovanović, Vitomir; Whitelock-Wainwright, Alexander; Gašević, Dragan; Dawson, Shane
• Is Part Of: Learning and instruction. Volume 72(2021)
• Publication Date: 2021-04
• Subjects: Feedback -- Learning analytics -- Self-regulated learning -- Large enrolment courses -- Higher education; Learning -- Periodicals; Teaching -- Periodicals; Apprentissage -- Périodiques; Enseignement -- Périodiques; Learning; Teaching; Periodicals; Electronic journals; 370.1
• DOI: 10.1016/j.learninstruc.2019.04.003
• Languages: English
• ISSNs: 0959-4752

Abstract

Abstract: Recent developments in educational technologies have provided a viable solution to the challenges associated with scaling personalised feedback to students. However, there is currently little empirical evidence about the impact such scaled feedback has on student learning progress and study behaviour. This paper presents the findings of a study that looked at the impact of a learning analytics (LA)-based feedback system on students' self-regulated learning and academic achievement in a large, first-year undergraduate course. Using the COPES model of self-regulated learning (SRL), we analysed the learning operations of students, by way of log data from the learning management system and e-book, as well as the products of SRL, namely, performance on course assessments, from three years of course offerings. The latest course offering involved an intervention condition that made use of an educational technology to provide LA-based process feedback. Propensity score matching was employed to match a control group to the student cohort enrolled in the latest course offering, creating two equal-sized groups of students who received the feedback (the experimental group) and those who did not (the control group). Growth mixture modelling and mixed between-within ANOVA were also employed to identify differences in the patterns of online self-regulated learning operations over the course of the semester. The results showed that the experimental group showed significantly different patterns in their learning operations and performed better in terms of final grades. Moreover, there was no difference in the effect of feedback on final grades among students with different prior academic achievement scores, indicating that the LA-based feedback deployed in this course is able to support students' learning, regardless of prior academic standing. Highlights: A learning analytics-based system was used to deliver process feedback to students in a course. The learning-analytics feedback employed multimodal data, such as log data from the learning management system and e-book. The pattern of self-regulated learning differed between students who had received the feedback, and those who had not. Final course marks were significantly higher for students who had received the feedback, compared to those who had not. There was no difference in impact of the LA-based, process feedback among students with different program entry scores.