Evaluation of machine learning techniques for forecast uncertainty quantification. (24th November 2022)

• Record Type: Journal Article
• Authors: Sacco, Maximiliano A.; Ruiz, Juan J.; Pulido, Manuel; Tandeo, Pierre
• Is Part Of: Quarterly journal of the Royal Meteorological Society. Volume 148:Number 749(2022)
• Page Start: 3470
• Page End: 3490
• Publication Date: 2022-11-24
• Subjects: chaotic dynamic models -- forecast -- neural networks -- observation likelihood loss function -- uncertainty quantification; Meteorology -- Periodicals; 551.5
• DOI: 10.1002/qj.4362
• Languages: English
• ISSNs: 0035-9009

Abstract

Abstract: Ensemble forecasting is, so far, the most successful approach to produce relevant forecasts with an estimation of their uncertainty. The main limitations of ensemble forecasting are the high computational cost and the difficulty to capture and quantify different sources of uncertainty, particularly those associated with model errors. In this article we perform toy‐model and state‐of‐the‐art model experiments to analyze to what extent artificial neural networks (ANNs) are able to model the different sources of uncertainty present in a forecast. In particular, those associated with the accuracy of the initial conditions and those introduced by the model error. We also compare different training strategies: one based on a direct training using the mean and spread of an ensemble forecast as target, and the other ones rely on an indirect training strategy using an analyzed state as target in which the uncertainty is implicitly learned from the data. Experiments using the Lorenz'96 model show that the ANNs are able to emulate some of the properties of ensemble forecasts like the filtering of the most unpredictable modes and a state‐dependent quantification of the forecast uncertainty. Moreover, ANNs provide a reliable estimation of the forecast uncertainty in the presence of model error. Preliminary experiments conducted with a state‐of‐the‐art forecasting system also confirm the ability of ANNs to produce a reliable quantification of the forecast uncertainty. Abstract : Proof‐of‐concept model experiments are conducted to examine the performance of artificial neural networks trained to predict a corrected state of the system and the state uncertainty using only a single deterministic forecast as input. We compare different training strategies: one based on a direct training using the mean and spread of an ensemble forecast as target, and the other ones rely on an indirect training strategy using a deterministic forecast as target in which the uncertainty is implicitly learned from the data.