Aggregating predictions from experts: A review of statistical methods, experiments, and applications. (16th June 2020)

• Record Type: Journal Article
• Authors: McAndrew, Thomas; Wattanachit, Nutcha; Gibson, Graham C.; Reich, Nicholas G.
• Is Part Of: Wiley interdisciplinary reviews. Volume 13:Number 2(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-06-16
• Subjects: consensus -- expert judgment -- forecast aggregation -- forecast combination -- judgmental forecasting; Mathematical statistics -- Data processing -- Periodicals; Science -- Data processing -- Periodicals; Social sciences -- Data processing -- Periodicals; Mathematical statistics -- Periodicals; 519.50285
• DOI: 10.1002/wics.1514
• Languages: English
• ISSNs: 1939-5108

Abstract

Abstract: Forecasts support decision making in a variety of applications. Statistical models can produce accurate forecasts given abundant training data, but when data is sparse or rapidly changing, statistical models may not be able to make accurate predictions. Expert judgmental forecasts—models that combine expert‐generated predictions into a single forecast—can make predictions when training data is limited by relying on human intuition. Researchers have proposed a wide array of algorithms to combine expert predictions into a single forecast, but there is no consensus on an optimal aggregation model. This review surveyed recent literature on aggregating expert‐elicited predictions. We gathered common terminology, aggregation methods, and forecasting performance metrics, and offer guidance to strengthen future work that is growing at an accelerated pace. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Multivariate Analysis Abstract : Expert judgmental forecasts—models that combine expert‐generated predictions into a single forecast—can make predictions when training data islimited and directly involve decision makers in the prediction process.We give an updated review of aggregating expert predictions in the digital age and recommendations for how to improve future work in this field.