Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions. (May 2015)

• Record Type: Journal Article
• Authors: Mellers, Barbara; Stone, Eric; Murray, Terry; Minster, Angela; Rohrbaugh, Nick; Bishop, Michael; Chen, Eva; Baker, Joshua; Hou, Yuan; Horowitz, Michael; Ungar, Lyle; Tetlock, Philip
• Is Part Of: Perspectives on psychological science. Volume 10:Number 3(2015)
• Page Start: 267
• Page End: 281
• Publication Date: 2015-05
• Subjects: predictions -- forecasts -- expertise -- teams -- probability training; Psychology -- Periodicals; 150
• DOI: 10.1177/1745691615577794
• Languages: English
• ISSNs: 1745-6916

Abstract

Across a wide range of tasks, research has shown that people make poor probabilistic predictions of future events. Recently, the U.S. Intelligence Community sponsored a series of forecasting tournaments designed to explore the best strategies for generating accurate subjective probability estimates of geopolitical events. In this article, we describe the winning strategy: culling off top performers each year and assigning them into elite teams of superforecasters . Defying expectations of regression toward the mean 2 years in a row, superforecasters maintained high accuracy across hundreds of questions and a wide array of topics. We find support for four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments. These findings suggest that superforecasters are partly discovered and partly created—and that the high-performance incentives of tournaments highlight aspects of human judgment that would not come to light in laboratory paradigms focused on typical performance.