GPU-accelerated DNS of compressible turbulent flows. (30th January 2023)

• Record Type: Journal Article
• Authors: Kim, Youngdae; Ghosh, Debojyoti; Constantinescu, Emil M.; Balakrishnan, Ramesh
• Is Part Of: Computers & fluids. Volume 251(2023)
• Publication Date: 2023-01-30
• Subjects: 76F65 -- 65Y05 -- 76F05 -- 35Q30 -- 65M06; Navier–Stokes equations -- GPUs -- WENO schemes -- Compressible flows -- Direct numerical simulation; Fluid dynamics -- Data processing -- Periodicals; 532.050285
• DOI: 10.1016/j.compfluid.2022.105744
• Languages: English
• ISSNs: 0045-7930

Abstract

Abstract: This paper explores strategies to transform an existing CPU-based high-performance computational fluid dynamics solver, HyPar, for compressible flow simulations on emerging exascale heterogeneous (CPU+GPU) computing platforms. The scientific motivation for developing a GPU-enhanced version of HyPar is to simulate canonical turbulent flows at the highest resolution possible on such platforms. We show that optimizing memory operations and thread blocks results in 200x speedup of computationally intensive kernels compared with a CPU core. Using multiple GPUs and CUDA-aware MPI communication, we demonstrate both strong and weak scaling of our GPU-based HyPar implementation on the NVIDIA Volta V100 GPUs. We simulate the decay of homogeneous isotropic turbulence in a triply periodic box on grids with up to 102 4 3 points (5.3 billion degrees of freedom) and on up to 1, 024 GPUs. We compare the wall times for CPU-only and CPU+GPU simulations. The results presented in the paper are obtained on the Summit and Lassen supercomputers at Oak Ridge and Lawrence Livermore National Laboratories, respectively. Highlights: Key strategies to accelerate CPU-based high-order finite-difference WENO scheme. GPU-accelerated algorithm for DNS of isotropic turbulence decay in a periodic box. Numerical justifications for our approaches and comparison to alternatives. 200x speedup using a GPU compared with a single CPU core. Scale-resolved simulations of turbulence on the Summit and Lassen supercomputers. 100 time steps with 5 billion degrees of freedom in less than 30 s using 1024 GPUs. Compare GPU and MPI simulations on the same number of nodes and show 10x speedup.