Benefits of Stochastic Weight Averaging in Developing Neural Network Radiation Scheme for Numerical Weather Prediction. (19th October 2022)

• Record Type: Journal Article
• Authors: Song, Hwan‐Jin; Roh, Soonyoung; Lee, Juho; Nam, Giung; Yun, Eunggu; Yoon, Jongmin; Kim, Park Sa
• Is Part Of: Journal of advances in modeling earth systems. Volume 14:Number 10(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-10-19
• Subjects: neural network -- stochastic weight averaging -- emulator -- speedup -- WRF -- RRTMG; Geological modeling -- Periodicals; Climatology -- Periodicals; Geochemical modeling -- Periodicals; 551.5011
• DOI: 10.1029/2021MS002921
• Languages: English
• ISSNs: 1942-2466

Abstract

Abstract: Stochastic weight averaging (SWA) was applied to improve the radiation emulator based on a sequential neural network (SNN) in a numerical weather prediction model over Korea. While the SWA has advantages in terms of generalization such as the ensemble model, the computational cost is maintained at the same level as that of a single model. In this study, the performances of both emulators were evaluated under ideal and real case frameworks. Various sensitivity experiments using different sampling ratios, activation functions, hidden layers, and batch sizes were also conducted. The emulators showed a 60‐fold speedup for the radiation processes and 84%–87% reduction of the total computation. In the ideal simulation, compared to the infrequent radiation scheme by 60 times, SNN improved forecast errors by 5.8%–14.1%, and SWA further increased these improvements by 18.2%–26.9%. In the real case simulation, SNN showed 8.8% and 4.7% improvements for longwave and shortwave (SW) fluxes compared to the infrequent method; however, these improvements decreased significantly after 5 days, resulting in 1.8% larger error for skin temperature. By contrast, SWA showed stable 1‐week forecast features with 12.6%, 8.0%, and 4.4% improvements in longwave and SW fluxes, and skin temperature, respectively. Although the use of two hidden layers showed the best performance in this study, it was thought that the optimal number of hidden layers could differ depending on the given problem. Compared to temperature and precipitation observations, all experiments showed a variability of error within 1%, implying that the operational use of the developed emulators is possible. Plain Language Summary: The neural network (NN) emulators for radiation parameterization have been actively developing to accelerate the computational speed of the numerical climate and weather forecasting models. Although previous studies have demonstrated that the computational speed for radiation processes can be improved tens of times, guaranteeing stability in long‐term forecasting has been recognized as imperative for the operational use of radiation emulator. In general, the multi‐model ensemble approach is used to reduce the uncertainty of a single model. However, this approach induces a significant computation burden in proportion to ensemble members. The alternative method developed in this study uses a stochastic averaging technique for weight coefficients during the NN training process, allowing processing to be conducted at the same computational cost as the single model because the dimensions of the final weights are maintained. Application of the trained NN emulator to the numerical weather forecasting model has demonstrated the advantages of generalization in various test cases, while exhibiting significant improvements in accuracy in the latter part of the forecast. This method can therefore contribute to improving emulator studies that face problems related to generalization. Key Points: The performance of the neural network (NN) radiation scheme was evaluated under ideal and real case frameworks Stochastic weight averaging is advantageous in generalization compared to the traditional NN Long‐term forecast errors can be largely improved using stochastic weight averaging