Monitoring trends in ensemble forecast performance focusing on surface variables and high‐impact events. (31st March 2019)

• Record Type: Journal Article
• Authors: Ben Bouallègue, Zied; Magnusson, Linus; Haiden, Thomas; Richardson, David S.
• Is Part Of: Quarterly journal of the Royal Meteorological Society. Volume 145:Number 721(2019)
• Page Start: 1741
• Page End: 1755
• Publication Date: 2019-03-31
• Subjects: climatology definition -- extreme events -- probabilistic forecast -- score comparison; Meteorology -- Periodicals; 551.5
• DOI: 10.1002/qj.3523
• Languages: English
• ISSNs: 0035-9009

Abstract

Abstract : This paper discusses methodological aspects of a monitoring process which focuses simultaneously on ensemble forecasts, surface variables, and high‐impact events. Which score(s) is (are) suitable for this task is a central question but not the only one to be answered. Here, we investigate the properties of the Brier score, logarithmic score, and diagonal elementary score in the context of forecast performance monitoring as well as the impact of methodological choices such as the event threshold definition, the reference forecast, and the role assigned to representativeness errors. A consistent picture of the verification process is eventually drawn where the design of the event climatology plays a key role. This study is illustrated by verification results for three surface variables (24 hr precipitation, 10 m wind speed, and 2 m temperature) over 15 years of operational ECMWF ensemble forecasting activities. Results are also compared with a current ECMWF headline score: the relative operating characteristic skill score for the Extreme Forecast Index. Abstract : Comparison of trends in IFS ensemble forecast performance for 24 hr precipitation using different probabilistic skill scores (rows) and different types of climatology definition (columns). The probabilistic skill scores are the Brier skill score (BSS), the logarithmic skill score (LSS), and the diagonal elementary skill score (DESS). The types of climatology definition are a long‐term climatology (left), a sample climatology (middle), and an eigenclimatology (right). Results for high‐impact events defined based on the 95th percentile of local climate distributions.