Development of NCL equivalent serial and parallel python']python routines for meteorological data analysis

被引:2
|
作者
Gharat, Jatin [1 ]
Kumar, Bipin [2 ]
Ragha, Leena [1 ]
Barve, Amit [3 ]
Jeelani, Shaik Mohammad [2 ]
Clyne, John [4 ]
机构
[1] Ramrao Adik Inst Technol, Comp Engn, Navi Mumbai, India
[2] Minist Earth Sci, Indian Inst Trop Meteorol, HPCS, Dr Homi Bhabha Rd, Pune 411008, Maharashtra, India
[3] Parul Univ, Parul Inst Engn & Technol, Comp Sci & Engn, Vadodara, Gujarat, India
[4] Natl Ctr Atmospher Res, Computat & Informat Syst Lab, POB 3000, Boulder, CO 80307 USA
关键词
National Center for Atmospheric Research command; language functions; !text type='python']python[!/text] routines; parallel [!text type='python']python[!/text] version; climate data analysis; high-performance computing; PERFORMANCE; MODELS;
D O I
10.1177/10943420221077110
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
The NCAR Command Language (NCL) is a popular scripting language used in the geoscience community for weather data analysis and visualization. Hundreds of years of data are analyzed daily using NCL to make accurate weather predictions. However, due to its sequential nature of execution, it cannot properly utilize the parallel processing power provided by High-Performance Computing systems (HPCs). Until now very few techniques have been developed to make use of the multi-core functionality of modern HPC systems on these functions. In the recent trend, open-source languages are becoming highly popular because they support major functionalities required for data analysis and parallel computing. Hence, developers of NCL have decided to adopt Python as the future scripting language for analysis and visualization and to enable the geosciences community to play an active role in its development and support. This study focuses on developing some of the widely used NCL routines in Python. To deal with the analysis of large datasets, parallel versions of these routines are developed to work within a single node and make use of multi-core CPUs to achieve parallelism. Results show high accuracy between NCL and Python outputs and the parallel versions provided good scaling compared to their sequential counterparts.
引用
收藏
页码:337 / 355
页数:19
相关论文
共 50 条
  • [31] The Space Physics Environment Data Analysis System in Python']Python
    Grimes, Eric W. W.
    Harter, Bryan
    Hatzigeorgiu, Nick
    Drozdov, Alexander
    Lewis, James W. W.
    Angelopoulos, Vassilis
    Cao, Xin
    Chu, Xiangning
    Hori, Tomo
    Matsuda, Shoya
    Jun, Chae-Woo
    Nakamura, Satoko
    Kitahara, Masahiro
    Segawa, Tomonori
    Miyoshi, Yoshizumi
    Le Contel, Olivier
    [J]. FRONTIERS IN ASTRONOMY AND SPACE SCIENCES, 2022, 9
  • [32] Building a scalable python']python distribution for HEP data analysis
    Lange, David J.
    [J]. 18TH INTERNATIONAL WORKSHOP ON ADVANCED COMPUTING AND ANALYSIS TECHNIQUES IN PHYSICS RESEARCH (ACAT2017), 2018, 1085
  • [33] yupi: Generation, tracking and analysis of trajectory data in Python']Python
    Reyes, A.
    Viera-Lopez, G.
    Morgado-Vega, J. J.
    Altshuler, E.
    [J]. ENVIRONMENTAL MODELLING & SOFTWARE, 2023, 163
  • [34] MPDAF - A Python']Python Package for the Analysis of VLT/MUSE Data
    Piqueras, L.
    Conseil, S.
    Shepherd, M.
    Bacon, R.
    Leclercq, F.
    Richard, J.
    [J]. ASTRONOMICAL DATA ANALYSIS SOFTWARE AND SYSTEMS XXVI, 2019, 521 : 545 - 548
  • [35] EMDA: A Python']Python package for Electron Microscopy Data Analysis
    Warshamanage, Rangana
    Yamashita, Keitaro
    Murshudov, Garib N.
    [J]. JOURNAL OF STRUCTURAL BIOLOGY, 2022, 214 (01)
  • [36] Development of a Python']Python-Based Algorithm for Comparative Analysis of Multiparticipant Next Generation Sequencing Data
    Martz, Flora G.
    Forst, Thomas M.
    Ryan, Sean M.
    Murphy, Patrick J. M.
    [J]. FASEB JOURNAL, 2016, 30
  • [37] School-Enterprise Cooperation on Python']Python Data Analysis Teaching
    Liu, Xinxin
    Xu, Hongyun
    [J]. 14TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND EDUCATION (ICCSE 2019), 2019, : 278 - 281
  • [38] ColiCoords: A Python']Python package for the analysis of bacterial fluorescence microscopy data
    Smit, Jochem H.
    Li, Yichen
    Warszawik, Eliza M.
    Herrmann, Andreas
    Cordes, Thorben
    [J]. PLOS ONE, 2019, 14 (06):
  • [39] pyerrors: A python']python framework for error analysis of Monte Carlo data
    Joswig, Fabian
    Kuberski, Simon
    Kuhlmann, Justus T.
    Neuendorf, Jan
    [J]. COMPUTER PHYSICS COMMUNICATIONS, 2023, 288
  • [40] A Study on Using Python']Python vs Weka on Dialysis Data Analysis
    Mitrpanont, Jarernsri
    Sawangphol, Wudhichart
    Vithantirawat, Thanita
    Paengkaew, Sinattaya
    Suwannasing, Prameyuda
    Daramas, Atthapan
    Chen, Yi-Cheng
    [J]. 2017 2ND INTERNATIONAL CONFERENCE ON INFORMATION TECHNOLOGY (INCIT), 2017, : 123 - 128