Beyond quadratic error: Case-study of a multiple criteria approach to the performance assessment of numerical forecasts of solar irradiance in the tropics. (January 2020)

• Record Type: Journal Article
• Authors: Verbois, Hadrien; Blanc, Philippe; Huva, Robert; Saint-Drenan, Yves-Marie; Rusydi, Andrivo; Thiery, Alexandre
• Is Part Of: Renewable & sustainable energy reviews. Volume 117(2020)
• Publication Date: 2020-01
• Subjects: Solar irradiance -- Solar forecasting -- Forecast verification -- WRF -- NWP -- Metrics -- Classification -- Frequency analysis -- Sensitivity analysis; Renewable energy sources -- Periodicals; Power resources -- Periodicals; Énergies renouvelables -- Périodiques; Ressources énergétiques -- Périodiques; 333.794
• DOI: 10.1016/j.rser.2019.109471
• Languages: English
• ISSNs: 1364-0321

Abstract

Abstract: As the penetration of photovoltaic power in the electrical grid increases, reliable forecasts of solar irradiance are becoming ever more critical. For forecasting horizons beyond a few hours, Numerical Weather Prediction models (NWP) are stapled to yield the best results. For potential users of NWP forecasts, choosing which model to use can be challenging, as a large palette of NWP models and configurations is available. To tackle this issue, several studies have conducted systematic comparisons between available models and configurations. Most of these studies, however, only evaluate forecast skills with a small number of metrics, such as the Root Mean Square Error (RMSE) or the Mean Average Error (MAE). To better understand the nature and specificities of various NWP models, we developed an evaluation approach that combines multiple-criteria analysis, Fourier analysis and classification of daily profiles of irradiance. In this paper, we present this novel approach and demonstrate its strengths by applying it to a practical use-case, using 108 different NWP forecasts models based on different parametrizations of the mesoscale NWP Weather Research and Forecasting model (WRF) in Singapore. Our approach allowed a clearer overview of the forecasts' skills, as well as better discrimination between models. It also significantly improved our understanding of the nature of the NWP errors. In particular, WRF forecasts in Singapore were found unfit to timely resolve irradiance at an hourly scale, but better adapted to predict daily profiles of irradiance. Furthermore, the proposed multiple criteria approach was applied to a sensitivity analysis of WRF physical schemes, providing a better insight into the impact of each model. This work shows that it is critical to go beyond RMSE when evaluating forecasts and that more holistic approaches such as the one proposed here should be preferred. Highlights: A novel multiple-criteria forecast evaluation approach was proposed. Multiple-criteria analysis gives a more comprehensive overview of forecasts skills. Multiple-criteria analysis also emphasizes differences between forecasting models. Fourier analysis leads to a better understanding of the nature of NWP forecasts. Ability to predict a daily profile of irradiance is a useful additional metric.