Performance and Accuracy Implications of Parallel Split Physics‐Dynamics Coupling in the Energy Exascale Earth System Atmosphere Model. (15th July 2020)

• Record Type: Journal Article
• Authors: Donahue, Aaron S.; Caldwell, Peter M.
• Is Part Of: Journal of advances in modeling earth systems. Volume 12:Number 7(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-07-15
• Subjects: computational performance -- parallel splitting -- physics‐dynamics coupling -- global atmosphere modeling -- time stepping -- numerical stability; Geological modeling -- Periodicals; Climatology -- Periodicals; Geochemical modeling -- Periodicals; 551.5011
• DOI: 10.1029/2020MS002080
• Languages: English
• ISSNs: 1942-2466

Abstract

Abstract: Simultaneous calculation of atmospheric processes is faster than calculating processes one at a time. This type of parallelism is beneficial or perhaps even necessary to provide good performance on modern supercomputers, which achieve faster performance through increased processor count rather than improved clock speed. The scalability of the Energy Exascale Earth System Model (E3SM) Atmosphere Model (EAM) is limited by the fluid dynamics which scales up to the number of mesh cells in the global mesh. In contrast, the suite of physics parameterizations in EAM is scalable up to the total number of physics columns, which is an order of magnitude greater than the number of mesh cells. A proposed solution to unlocking the greater potential performance from the physics suite is to solve the physics and dynamics in parallel. This work represents a first attempt at parallel splitting of the grid‐scale fluid dynamics model and the subgrid‐scale physics parameterizations in a global atmosphere model. We will demonstrate that switching to parallel physics‐dynamics coupling extends the scalability of the EAM to up to 3 times the previous peak scalability limit and is up to 20% faster than the sequentially split coupling at the highest core counts and the same time step. Decadal simulations of both coupling approaches show very little impact to the model climate. This improved performance does not come without drawbacks, however. Parallel splitting requires a shorter time step and other modifications which largely offset performance gains. A mass fixer is required for conservation. Techniques for mitigating these issues are also discussed. Plain Language Summary: In order to improve the computational performance of global atmosphere models, the community must look at the impact of switching to more parallel splitting of individual processes, thus increasing the amount of work that can be distributed over a large number of computational units. This work looks at the impact to both accuracy and computational performance when the two most expensive components in the Energy Exascale Earth System Model (E3SM) atmosphere model (EAM) are coupled in parallel instead of sequentially. The results are mixed: Model climate does not appear to be impacted by the switch in coupling; however, the performance gains were modest and fall short of the predicted improvements. This work discusses exactly why the performance gains fell short and points out the major infrastructure changes that would be required to extract the maximum performance gains. The results here will be helpful to any effort to adopt parallel splitting in any multiphysics model. Key Points: Parallel coupling of physics and dynamics improves model scalability Mass conservation and stability are major concerns for parallel splitting implementation Parallel splitting may require a more restrictive model time step