Adoption and impact of a learning analytics dashboard supporting the advisor—Student dialogue in a higher education institute in Latin America. (4th June 2020)

• Record Type: Journal Article
• Authors: De Laet, Tinne; Millecamp, Martijn; Ortiz‐Rojas, Margarita; Jimenez, Alberto; Maya, Ricardo; Verbert, Katrien
• Is Part Of: British journal of educational technology. Volume 51:Number 4(2020:Jul.)
• Page Start: 1002
• Page End: 1018
• Publication Date: 2020-06-04
• Subjects: Academic Advising -- Case Study -- Impact -- Learning Analytics -- Learning Dashboards -- Learning Technology; Audio-visual education -- Periodicals; 371.33
• DOI: 10.1111/bjet.12962
• Languages: English
• ISSNs: 0007-1013

Abstract

Abstract : This paper presents a case study on the adoption and the impact of new modules in a learning analytics dashboard supporting the dialogue between student advisors and students when advising on a study plan for the next academic semester in Escuela Superior Politecnica del Litoral, a higher education institute in Ecuador. The impact and the adoption of the new dashboard modules were assessed using a mixed‐methods approach. The quantitative approach builds on data of 172 advisors in 34 programs and 4481 advising sessions in 2019 (post) and 4747 advising sessions in 2018 (pre) to assess the adoption and use of the dashboard, the level of support experienced by the advisors, the impact of the new dashboard modules on the difference between the advised study plan and the plan students register for, and students' academic achievement. The qualitative approach with observations of 14 staged advising dialogues and semi‐structured interviews with eight advisors was used to assess how the dashboard was used and to get deeper understanding of the perceived usefulness and impact of the dashboard. The results show that an institution‐wide deployment of dashboard modules tailored to the needs of the advisors can be achieved and can increase the level of support perceived by the advisors and significantly decrease the gap between the suggested study plans in advising dialogues and the study plans that students actually register for. On the short‐term, however, no significant changes in academic achievement were observed. Practitioner Notes What is already known about this topic? Academic advising can positively impact retention, academic achievement and study completion. Learning analytics dashboards are promising pieces of educational technology for academic advising as they can trigger reflection and sense‐making of educational data. Evaluation of learning analytics dashboards is often still immature and not well‐connected to the actual goals of the dashboards. Large‐scale evaluations looking at impact of dashboards are even scarcer. What this paper adds? This paper adds, to the scarce scientific evidence on academic advising dashboards, a large‐scale case study on a dashboard supporting the advisor student dialogue during the composition of well‐balanced study plans. The paper presents research evidence of the impact of the dashboard on the support advisors experience, the study plans suggested by the advisors and the ones actually registered by the students and students' academic achievement. Evidence is based on a quantitative analysis, using data of 172 advisors from 34 programs representing more than 9000 advising dialogues, and a qualitative analysis using observations and interviews. Implications for practice and/or policy Dashboards to support academic advising dialogues can be realized institution‐wide at scale. Training of student advisors supports a large scale deployment. Well‐designed dashboards that focus on addressing needs of advisors increase the level of support that advisor experience when advising students. Dashboard accommodating the simulation of study plans and the workload associated with them, succeed in decreasing the variance in suggested plans between advisors and reduce the gap between the study plans that advisors suggest to student and the study plans that students actually register for. Short‐term impact on academic achievement was not observed.