Modelling Large-Scale Scientific Data Transfers

被引：0
|
作者：
Bogado J. [1 ,2 ]
Lassnig M. [3 ]
Monticelli F. [2 ]
Díaz J. [1 ]
机构：
[1] LINTI, Facultad de Informática, La Plata
[2] IFLP, UNLP, CONICET, La Plata
[3] European Organization for Nuclear Research (CERN), Geneva
来源：
Computing and Software for Big Science | 2022年 / 6卷 / 1期
关键词：
Data transfer analysis; Distributed computing modelling; Performance metrics;
D O I：
10.1007/s41781-022-00084-4
中图分类号：
学科分类号：
摘要：
This work focuses on the study of a recently published dataset (Bogado et al. in ATLAS Rucio transfers dataset. Zenodo, 2020.) with data that allow us to reconstruct the lifetime of file transfers in the contexts of the Worldwide LHC Computing Grid (WLCG). Several models for Rule Time To Complete (TTC) prediction are presented and evaluated. The dataset source is Rucio, an open-source software framework that provides scientific collaborations with the functionality to organize, manage, and access their data at scale. The rich amount of data gathered about the transfers and rules, presents a unique opportunity to better understand the complex mechanisms involved in file transfers across the WLCG. © 2022, The Author(s).
引用
收藏
相关论文
共 50 条
  • [1] Disinformative data in large-scale hydrological modelling
    Kauffeldt, A.
    Halldin, S.
    Rodhe, A.
    Xu, C. -Y.
    Westerberg, I. K.
    [J]. HYDROLOGY AND EARTH SYSTEM SCIENCES, 2013, 17 (07) : 2845 - 2857
  • [2] Mesh data management in large-scale scientific computing
    Chen, Hong
    Zheng, Winmin
    [J]. PROCEEDINGS OF THE THIRD CHINAGRID ANNUAL CONFERENCE, 2008, : 144 - 152
  • [3] Parallel Tensor Compression for Large-Scale Scientific Data
    Austin, Woody
    Ballard, Grey
    Kolda, Tamara G.
    [J]. 2016 IEEE 30TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS 2016), 2016, : 912 - 922
  • [4] Parallel visualization of large-scale multifield scientific data
    Cao, Yi
    Mo, Zeyao
    Ai, Zhiwei
    Wang, Huawei
    Xiao, Li
    Zhang, Zhe
    [J]. JOURNAL OF VISUALIZATION, 2019, 22 (06) : 1107 - 1123
  • [5] Understanding Data Similarity in Large-Scale Scientific Datasets
    Linton, Payton
    Melodia, William
    Lazar, Alina
    Agarwal, Deborah
    Bianchi, Ludovico
    Ghoshal, Devarshi
    Pastorello, Gilbert
    Ramakrishnan, Lavanya
    Wu, Kesheng
    [J]. 2019 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2019, : 4525 - 4531
  • [6] Parallel visualization of large-scale multifield scientific data
    Yi Cao
    Zeyao Mo
    Zhiwei Ai
    Huawei Wang
    Li Xiao
    Zhe Zhang
    [J]. Journal of Visualization, 2019, 22 : 1107 - 1123
  • [7] Modeling and Optimizing Large-Scale Wide-Area Data Transfers
    Kettimuthu, Rajkumar
    Vardoyan, Gayane
    Agrawal, Gagan
    Sadayappan, P.
    [J]. 2014 14TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID), 2014, : 196 - 205
  • [8] A survey of the techniques of volume rendering for large-scale scientific data
    Wang, Huawei
    He, Liu
    Cao, Yi
    Xiao, Li
    [J]. Guofang Keji Daxue Xuebao/Journal of National University of Defense Technology, 2020, 42 (02): : 1 - 12
  • [9] Network Adaptation for Large Scientific Data Transfers
    Zivkovic, M.
    Zeljkovic, V.
    [J]. 2016 GLOBAL MEDICAL ENGINEERING PHYSICS EXCHANGES/PAN AMERICAN HEALTH CARE EXCHANGES (GMEPE/PAHCE), 2016,
  • [10] An assessment of large-scale flood modelling based on LiDAR data
    Chone, Guenole
    Biron, Pascale M.
    Buffin-Belanger, Thomas
    Mazgareanu, Iulia
    Neal, Jeff C.
    Sampson, Christopher C.
    [J]. HYDROLOGICAL PROCESSES, 2021, 35 (08)
12345