When to consult precision-recall curves. (March 2020)

• Record Type: Journal Article
• Authors: Cook, Jonathan; Ramadas, Vikram
• Is Part Of: Stata journal. Volume 20:Number 1(2020)
• Page Start: 131
• Page End: 148
• Publication Date: 2020-03
• Subjects: st0591 -- prcurve -- precision-recall curves -- classifier evaluation -- ROC curves; Statistics -- Periodicals; Statistics -- Computer programs -- Periodicals; 001.422
• DOI: 10.1177/1536867X20909693
• Languages: English
• ISSNs: 1536-867X

Abstract

Receiver operating characteristic (ROC) curves are commonly used to evaluate predictions of binary outcomes. When there is a small percentage of items of interest (as would be the case with fraud detection, for example), ROC curves can provide an inflated view of performance. This can cause challenges in determining which set of predictions is better. In this article, we discuss the conditions under which precision-recall curves may be preferable to ROC curves. As an illustrative example, we compare two commonly used fraud predictors (Beneish's [1999, Financial Analysts Journal 55: 24–36] M score and Dechow et al.'s [2011, Contemporary Accounting Research 28: 17–82] F score) using both ROC and precision-recall curves. To aid the reader with using precision-recall curves, we also introduce the commandprcurve to plot them.