Development of NCL equivalent serial and parallel python']python routines for meteorological data analysis

被引:2
|
作者
Gharat, Jatin [1 ]
Kumar, Bipin [2 ]
Ragha, Leena [1 ]
Barve, Amit [3 ]
Jeelani, Shaik Mohammad [2 ]
Clyne, John [4 ]
机构
[1] Ramrao Adik Inst Technol, Comp Engn, Navi Mumbai, India
[2] Minist Earth Sci, Indian Inst Trop Meteorol, HPCS, Dr Homi Bhabha Rd, Pune 411008, Maharashtra, India
[3] Parul Univ, Parul Inst Engn & Technol, Comp Sci & Engn, Vadodara, Gujarat, India
[4] Natl Ctr Atmospher Res, Computat & Informat Syst Lab, POB 3000, Boulder, CO 80307 USA
关键词
National Center for Atmospheric Research command; language functions; !text type='python']python[!/text] routines; parallel [!text type='python']python[!/text] version; climate data analysis; high-performance computing; PERFORMANCE; MODELS;
D O I
10.1177/10943420221077110
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
The NCAR Command Language (NCL) is a popular scripting language used in the geoscience community for weather data analysis and visualization. Hundreds of years of data are analyzed daily using NCL to make accurate weather predictions. However, due to its sequential nature of execution, it cannot properly utilize the parallel processing power provided by High-Performance Computing systems (HPCs). Until now very few techniques have been developed to make use of the multi-core functionality of modern HPC systems on these functions. In the recent trend, open-source languages are becoming highly popular because they support major functionalities required for data analysis and parallel computing. Hence, developers of NCL have decided to adopt Python as the future scripting language for analysis and visualization and to enable the geosciences community to play an active role in its development and support. This study focuses on developing some of the widely used NCL routines in Python. To deal with the analysis of large datasets, parallel versions of these routines are developed to work within a single node and make use of multi-core CPUs to achieve parallelism. Results show high accuracy between NCL and Python outputs and the parallel versions provided good scaling compared to their sequential counterparts.
引用
收藏
页码:337 / 355
页数:19
相关论文
共 50 条
  • [1] MetPy: A Meteorological Python']Python Library for Data Analysis and Visualization
    May, Ryan M.
    Goebbert, Kevin H.
    Thielen, Jonathan E.
    Leeman, John R.
    Camron, M. Drew
    Bruick, Zachary
    Bruning, Eric C.
    Manser, Russell P.
    Arms, Sean C.
    Marsh, Patrick T.
    [J]. BULLETIN OF THE AMERICAN METEOROLOGICAL SOCIETY, 2022, 103 (10) : E2273 - E2284
  • [2] Climate analysis routines using Python']Python
    Sáenz, J
    Zubillaga, J
    [J]. DEVELOPMENT AND APPLICATION OF COMPUTER TECHNIQUES TO ENVIRONMENTAL STUDIES VIII, 2000, 4 : 279 - 287
  • [3] SERIAL AND DIFFERENT PARALLEL IMPLEMENTATIONS OF LATTICE BOLTZMANN METHOD IN PYTHON']PYTHON: A COMPARATIVE ANALYSIS
    Ajrian, Ashkan
    Ebadi, Mohammad
    Delijani, Ebrahim Biniaz
    Koroteev, Dmitry
    [J]. COMPUTATIONAL THERMAL SCIENCES, 2023, 15 (05): : 55 - 70
  • [4] Analysis of counting data: Development of the SATLAS Python']Python package
    Gins, W.
    de Groote, R. P.
    Bissell, M. L.
    Buitrago, C. Granados
    Ferrer, R.
    Lynch, K. M.
    Neyens, G.
    Sels, S.
    [J]. COMPUTER PHYSICS COMMUNICATIONS, 2018, 222 : 286 - 294
  • [5] Performance Analysis of Parallel Python']Python Applications
    Wagner, Michael
    Llort, German
    Mercadal, Estanislao
    Gimenez, Judit
    Labarta, Jesus
    [J]. INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE (ICCS 2017), 2017, 108 : 2171 - 2179
  • [6] A Python']Python package based on robust statistical analysis for serial crystallography data processing
    Hadian-Jazi, Marjan
    Sadri, Alireza
    [J]. ACTA CRYSTALLOGRAPHICA SECTION D-STRUCTURAL BIOLOGY, 2023, 79 : 820 - 829
  • [7] Application of Python']Python Parallel Computing in Online Identification of Thevenin Equivalent Parameters
    Meng Xinyuan
    Wen Tao
    Ma Kaigang
    [J]. 2020 IEEE STUDENT CONFERENCE ON ELECTRIC MACHINES AND SYSTEMS (SCEMS 2020), 2020, : 20 - 23
  • [8] Teaching Parallel Computing and Dependence Analysis with Python']Python
    Watkinson, Neftali
    Shivam, Aniket
    Nicolau, Alexandru
    Veidenbaum, Alexander V.
    [J]. 2019 IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM WORKSHOPS (IPDPSW), 2019, : 320 - 325
  • [9] nbodykit: A Python']Python Toolkit for Cosmology Simulations and Data Analysis on Parallel HPC Systems
    Hand, Nick
    Feng, Yu
    [J]. PROCEEDINGS OF PYHPC'17: 7TH WORKSHOP ON PYTHON FOR HIGH-PERFORMANCE AND SCIENTIFIC COMPUTING, 2017,
  • [10] PTRAIL - A python']python package for parallel trajectory data preprocessing
    Haidri, Salman
    Haranwala, Yaksh J.
    Bogorny, Vania
    Renso, Chiara
    da Fonseca, Vinicius Prado
    Soares, Amilcar
    [J]. SOFTWAREX, 2022, 19