Simple Rules to Guide Expert Classifications. (27th May 2020)

• Record Type: Journal Article
• Authors: Jung, Jongbin; Concannon, Connor; Shroff, Ravi; Goel, Sharad; Goldstein, Daniel G.
• Is Part Of: Journal of the Royal Statistical Society. Volume 183:Number 3(2020)
• Page Start: 771
• Page End: 800
• Publication Date: 2020-05-27
• Subjects: Heuristics -- Judgement and decision making -- Policy evaluation -- Sensitivity analysis; Social sciences -- Statistical methods -- Periodicals; Statistics -- Periodicals; 300.15195
• DOI: 10.1111/rssa.12576
• Languages: English
• ISSNs: 0964-1998

Abstract

Summary: Judges, doctors and managers are among those decision makers who must often choose a course of action under limited time, with limited knowledge and without the aid of a computer. Because data-driven methods typically outperform unaided judgements, resource-constrained practitioners can benefit from simple, statistically derived rules that can be applied mentally. In this work, we formalize long-standing observations about the efficacy of improper linear models to construct accurate yet easily applied rules. To test the performance of this approach, we conduct a large-scale evaluation in 22 domains and focus in detail on one: judicial decisions to release or detain defendants while they await trial. In these domains, we find that simple rules rival the accuracy of complex prediction models that base decisions on considerably more information. Further, comparing with unaided judicial decisions, we find that simple rules substantially outperform the human experts. To conclude, we present an analytical framework that sheds light on why simple rules perform as well as they do.