A Big Data and Learning Analytics Approach to Process-Level Feedback in Cognitive Simulations. (February 2017)

• Record Type: Journal Article
• Authors: Pecaric, Martin; Boutis, Kathy; Beckstead, Jason; Pusic, Martin
• Is Part Of: Academic medicine. Volume 92:Number 2(2017)
• Publication Date: 2017-02
• Subjects: Medical education -- Periodicals; Medical policy -- Periodicals; Medical personnel -- Periodicals; Periodicals; 610.711
• DOI: 10.1097/ACM.0000000000001234
• Languages: English
• ISSNs: 1040-2446

Abstract

Abstract : Collecting and analyzing large amounts of process data for the purposes of education can be considered a big data/learning analytics (BD/LA) approach to improving learning. However, in the education of health care professionals, the application of BD/LA is limited to date. The authors discuss the potential advantages of the BD/LA approach for the process of learning via cognitive simulations. Using the lens of a cognitive model of radiograph interpretation with four phases (orientation, searching/scanning, feature detection, and decision making), they reanalyzed process data from a cognitive simulation of pediatric ankle radiography where 46 practitioners from three expertise levels classified 234 cases online. To illustrate the big data component, they highlight the data available in a digital environment (time-stamped, click-level process data). Learning analytics were illustrated using algorithmic computer-enabled approaches to process-level feedback. For each phase, the authors were able to identify examples of potentially useful BD/LA measures. For orientation, the trackable behavior of re-reviewing the clinical history was associated with increased diagnostic accuracy. For searching/scanning, evidence of skipping views was associated with an increased false-negative rate. For feature detection, heat maps overlaid on the radiograph can provide a metacognitive visualization of common novice errors. For decision making, the measured influence of sequence effects can reflect susceptibility to bias, whereas computer-generated path maps can provide insights into learners' diagnostic strategies. In conclusion, the augmented collection and dynamic analysis of learning process data within a cognitive simulation can improve feedback and prompt more precise reflection on a novice clinician's skill development. Abstract : Supplemental Digital Content is available in the text.