Improving visual communication of discriminative accuracy for predictive models: the probability threshold plot. Issue 1 (12th March 2021)

• Record Type: Journal Article
• Authors: Johnston, Stephen S; Fortin, Stephen; Kalsekar, Iftekhar; Reps, Jenna; Coplan, Paul
• Is Part Of: JAMIA open. Volume 4:Issue 1(2021)
• Publication Date: 2021-03-12
• Subjects: predictive analytics -- discriminative accuracy -- receiver operating characteristic curve; Medical informatics -- Periodicals; 610.285
• DOI: 10.1093/jamiaopen/ooab017
• Languages: English
• ISSNs: 2574-2531

Abstract

Abstract: Objectives: To propose a visual display—the probability threshold plot (PTP)—that transparently communicates a predictive models' measures of discriminative accuracy along the range of model-based predicted probabilities ( Pt ). Materials and Methods: We illustrate the PTP by replicating a previously-published and validated machine learning-based model to predict antihyperglycemic medication cessation within 1–2 years following metabolic surgery. The visual characteristics of the PTPs for each model were compared to receiver operating characteristic (ROC) curves. Results: A total of 18 887 patients were included for analysis. Whereas during testing each predictive model had nearly identical ROC curves and corresponding area under the curve values (0.672 and 0.673), the visual characteristics of the PTPs revealed substantive between-model differences in sensitivity, specificity, PPV, and NPV across the range of Pt . Discussion and Conclusions: The PTP provides improved visual display of a predictive model's discriminative accuracy, which can enhance the practical application of predictive models for medical decision making.