Development of NCL equivalent serial and parallel python routines for meteorological data analysis. (May 2022)

• Record Type: Journal Article
• Authors: Gharat, Jatin; Kumar, Bipin; Ragha, Leena; Barve, Amit; Jeelani, Shaik Mohammad; Clyne, John
• Is Part Of: International journal of high performance computing applications. Volume 36:Number 3(2022)
• Page Start: 337
• Page End: 355
• Publication Date: 2022-05
• Subjects: National Center for Atmospheric Research command -- language functions -- python routines -- parallel python version -- climate data analysis -- high-performance computing; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/10943420221077110
• Languages: English
• ISSNs: 1094-3420

Abstract

The NCAR Command Language (NCL) is a popular scripting language used in the geoscience community for weather data analysis and visualization. Hundreds of years of data are analyzed daily using NCL to make accurate weather predictions. However, due to its sequential nature of execution, it cannot properly utilize the parallel processing power provided by High-Performance Computing systems (HPCs). Until now very few techniques have been developed to make use of the multi-core functionality of modern HPC systems on these functions. In the recent trend, open-source languages are becoming highly popular because they support major functionalities required for data analysis and parallel computing. Hence, developers of NCL have decided to adopt Python as the future scripting language for analysis and visualization and to enable the geosciences community to play an active role in its development and support. This study focuses on developing some of the widely used NCL routines in Python. To deal with the analysis of large datasets, parallel versions of these routines are developed to work within a single node and make use of multi-core CPUs to achieve parallelism. Results show high accuracy between NCL and Python outputs and the parallel versions provided good scaling compared to their sequential counterparts.