Measuring leadership development in workplace learning using automated assessments: Learning analytics and measurement theory approach. (31st March 2022)

• Record Type: Journal Article
• Authors: Barthakur, Abhinava; Kovanovic, Vitomir; Joksimovic, Srecko; Zhang, Zhonghua; Richey, Michael; Pardo, Abelardo
• Is Part Of: British journal of educational technology. Volume 53:Number 6(2022)
• Page Start: 1842
• Page End: 1863
• Publication Date: 2022-03-31
• Subjects: cognitive diagnostic models -- leadership measurement -- learning analytics -- self‐reflection; Audio-visual education -- Periodicals; 371.33
• DOI: 10.1111/bjet.13218
• Languages: English
• ISSNs: 0007-1013

Abstract

Abstract: Technological affordances have shown promising potential in advancing the delivery of corporate learning programmes designed for professional leadership development. However, there is a considerable challenge in evaluating learners' skill acquisition, with most of the past research relying on pre‐ and post‐tests or other forms of self‐reports to measure leadership development. In that sense, these approaches measure leadership development before and after the programme, while being inefficient for measuring the development during the learning process. This study collected self‐reflection answers from a professional development MOOC that allows learners to express their stepwise learning and reflect on their professional experience on leadership fronts. We developed a novel methodology and an automated system for the evaluation of leadership skills' mastery based on the depth of reflection exhibited during the learning process. We identified four groups of learners based on their course content mastery and explored the differences within groups. The results also highlight relevant insights about instructional design and provide promising avenues for future research. Practitioner notes What is already known about this topic Professional leadership programmes have become increasingly common in workplace learning. Programmes mainly use manual/introspective measures to assess skill acquisition. What this paper adds An automated assessment system to evaluate leadership skill mastery. Evidence‐based and leadership driven inferences about skill acquisition. Use of a novel multidisciplinary methodology for complex skills assessment. Implications of practice and/or policy Assessing leadership development should include more than course grades. Assessing differences in content mastery requires evaluation of various skills. Developed assessment system provides promise for other similar domains.