Evaluating a Prototype of a Recommender‐Driven Online Learning System. (15th October 2018)

• Record Type: Journal Article
• Authors: Peiris, K. Dharini Amitha; Gallupe, R. Brent
• Is Part Of: Decision sciences journal of innovative education. Volume 16:Number 4(2018:Oct.)
• Page Start: 281
• Page End: 309
• Publication Date: 2018-10-15
• Subjects: Design science research -- Formative and summative evaluations -- Moderate constructivist learning -- and Recommender‐driven online learning systems; Business education -- Periodicals; 658.00711
• DOI: 10.1111/dsji.12164
• Languages: English
• ISSNs: 1540-4595

Abstract

ABSTRACT: Recommender‐driven online learning systems (ROLS) are at the forefront of new computer‐based learning. They incorporate machine learning to allow learning‐by‐doing, generating personalized recommendations in the process. This article describes the evaluations of a new type of online learning systems, ROLS. This evaluation was carried out in three phases using a design science research approach. In Phase I, building the ROLS prototype validated the conceptual framework used. In Phase II, building an instantiation of ROLS to teach Structured Query Language (SQL), SQL‐with‐Ease, validated the ROLS prototype. In Phase III, a laboratory experiment evaluated learning outcomes from using SQL‐with‐Ease compared with two other traditional forms of learning. A set of qualitative interviews carried out with learners soon after using the system confirmed that the system was effective. They indicated that more work on fine‐tuning recommendations generated by the system could further improve learner satisfaction. The key implication for practitioners is that ROLS have the potential to improve learning outcomes significantly. Implications for researchers are that evaluations of ROLS, which include formative and summative evaluations, are essential to improve their performance and that developing innovative approaches to evaluation can advance these learning technologies.