On the Need to Automate Support for Quality Assessment Studies of Space Weather Models. Issue 11 (8th November 2018)

• Record Type: Journal Article
• Authors: MacNeice, P.
• Is Part Of: Space weather. Volume 16:Issue 11(2018)
• Page Start: 1627
• Page End: 1634
• Publication Date: 2018-11-08
• Subjects: automated model validation -- space weather -- ambient solar wind; Space environment -- Periodicals; 551.509992
• DOI: 10.1029/2018SW002039
• Languages: English
• ISSNs: 1542-7390

Abstract

Abstract: Model quality assessment (QA) and, where possible, model validation are critical steps in the life cycle of models destined to play a role in space weather research and forecasting. A major goal of QA is to characterize the state of the art of models of a given class, for example, models of the global corona and solar wind. This requires consistent assessment of all major models in the class. Commonly, QA has been done in what amounts to campaign mode, generating reports that capture the quality of a particular model or a limited set of models for specific events or time intervals. Inconsistencies between study designs, the limited scope of each study, and the intermittency with which the study reports appear in the literature mean that we never achieve a complete assessment of the state of the art. In addition, the time frame in which new models appear or existing models are upgraded is comparable to the publication time of peer‐reviewed validation reports, which means that these reports are often out of date soon after they are published. The community's current QA strategy is inadequate and unsustainable. In this paper we show why it is unsustainable and advocate for the development of automated protocols which can, with minimal ongoing labor cost, support the community's efforts to maintain up‐to‐date results. We illustrate the concept with results from a pilot scheme developed in conjunction with the Solar Heliospheric and Interplanetary Environment community. Key Points: Automating model quality assessment and validation is an important goal A prototype automated quality assessment and validation system has been developed It has been applied to models of the ambient solar wind