The importance of appropriate verification metrics for the assessment of observation impact in a convection‐permitting modelling system. (11th September 2018)

• Record Type: Journal Article
• Authors: Necker, Tobias; Weissmann, Martin; Sommer, Matthias
• Is Part Of: Quarterly journal of the Royal Meteorological Society. Volume 144:Number 714(2018)
• Page Start: 1667
• Page End: 1680
• Publication Date: 2018-09-11
• Subjects: convective‐scale -- data assimilation -- EFSOI -- forecast sensitivity to observations -- FSOI -- observation impact -- verification metric; Meteorology -- Periodicals; 551.5
• DOI: 10.1002/qj.3390
• Languages: English
• ISSNs: 0035-9009

Abstract

Abstract : Over the past 15 years, adjoint‐based, ensemble‐based and hybrid methods have been developed for estimating observation impact based on the forecast sensitivity to observation impact (FSOI). These methods are now commonly used in global modelling systems. However, little attention has been given to assessing observation impact in regional convection‐permitting modelling systems. This study presents the first evaluation of ensemble‐based estimates of observation impact over an extended period of six weeks in such a convection‐permitting modelling system, namely the regional ensemble system of Deutscher Wetterdienst. Another aspect that has received little attention is the choice of the forecast‐error verification metric. Nearly all previous studies used the difference between the forecast and a subsequent analysis of the same modelling system expressed in terms of energy (total energy norm). While such a self‐verification generally needs to be treated with caution, it appears unsuitable for convection‐permitting regional forecasts. Firstly, total energy does not really reflect parameters that forecast users are interested in, and important forecast quantities such as surface wind gusts and precipitation are not even part of the analysis. Secondly, systematic analysis and forecast errors are non‐negligible in the presence of convection, especially for important variables that are related to convection. To overcome this issue, we introduce the use of independent radar observations for the verification of observation impact and compare results for a variety of different observation‐based metrics for a six‐week high‐impact weather period in summer 2016. This revealed a particular sensitivity of the estimated impact to model as well as observation biases and sensitivity studies indicated that even small biases can have an influence on the estimated impact. Additionally, we demonstrate that FSOI can be used to identify biases through comparison of results for different metrics. Abstract : This study emphasizes the importance of appropriate verification metrics to achieve a reliable assessment of observation impact. A new verification metric based on independent radar observations is presented and applied to assess observation impact on the convective‐scale regional ensemble data assimilation system of Deutscher Wetterdienst. The figure shows the KENDA/COSMO‐DE domain and spatial distribution of the assimilated observation types at 1200 UTC on 29 May 2016, showing aircraft (AIREP, triangles), wind profiler (PROF, squares), surface station (SYNOP, circles) and radiosonde (TEMP, diamonds) observations.