Modelling Large-Scale Scientific Data Transfers

被引:0
|
作者
Bogado J. [1 ,2 ]
Lassnig M. [3 ]
Monticelli F. [2 ]
Díaz J. [1 ]
机构
[1] LINTI, Facultad de Informática, La Plata
[2] IFLP, UNLP, CONICET, La Plata
[3] European Organization for Nuclear Research (CERN), Geneva
关键词
Data transfer analysis; Distributed computing modelling; Performance metrics;
D O I
10.1007/s41781-022-00084-4
中图分类号
学科分类号
摘要
This work focuses on the study of a recently published dataset (Bogado et al. in ATLAS Rucio transfers dataset. Zenodo, 2020.) with data that allow us to reconstruct the lifetime of file transfers in the contexts of the Worldwide LHC Computing Grid (WLCG). Several models for Rule Time To Complete (TTC) prediction are presented and evaluated. The dataset source is Rucio, an open-source software framework that provides scientific collaborations with the functionality to organize, manage, and access their data at scale. The rich amount of data gathered about the transfers and rules, presents a unique opportunity to better understand the complex mechanisms involved in file transfers across the WLCG. © 2022, The Author(s).
引用
收藏
相关论文
共 50 条
  • [41] Optimizing data query performance of Bi-cluster for large-scale scientific data in supercomputers
    Xia Liao
    Yixian Shen
    Shengguo Li
    Yutong Lu
    Yufei Du
    Zhiguang Chen
    [J]. The Journal of Supercomputing, 2022, 78 : 2417 - 2441
  • [42] Optimizing data query performance of Bi-cluster for large-scale scientific data in supercomputers
    Liao, Xia
    Shen, Yixian
    Li, Shengguo
    Lu, Yutong
    Du, Yufei
    Chen, Zhiguang
    [J]. JOURNAL OF SUPERCOMPUTING, 2022, 78 (02): : 2417 - 2441
  • [43] LARGE-SCALE SCIENTIFIC COMPUTATION VIA MINICOMPUTER
    SCHAEFER, HF
    MILLER, WH
    [J]. COMPUTERS & CHEMISTRY, 1977, 1 (02): : 85 - 90
  • [44] Fault tolerance in large-scale scientific computing
    Hough, Patricia D.
    Howle, Victoria E.
    [J]. PARALLEL PROCESSING FOR SCIENTIFIC COMPUTING, 2006, : 203 - 220
  • [45] Java']Java for large-scale scientific computations?
    Krall, A
    Tomsich, P
    [J]. LARGE-SCALE SCIENTIFIC COMPUTING, 2001, 2179 : 228 - 235
  • [46] A methodology for scientific benchmarking with large-scale applications
    Armstrong, B
    Eigenmann, R
    [J]. PERFORMANCE EVALUATION AND BENCHMARKING WITH REALISTIC APPLICATIONS, 2001, : 109 - 127
  • [47] Usage Behavior of a Large-Scale Scientific Archive
    Adams, Ian F.
    Madden, Brian A.
    Frank, Joel C.
    Storer, Mark W.
    Miller, Ethan L.
    Harano, Gene
    [J]. 2012 INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS (SC), 2012,
  • [48] Rapid modelling of the large-scale magnetospheric field from Swarm satellite data
    Brian Hamilton
    [J]. Earth, Planets and Space, 2013, 65 : 1295 - 1308
  • [49] Modelling and developing conflict-aware scheduling on large-scale data centres
    Wang, Bin
    Chen, Chao
    He, Ligang
    Gao, Bo
    Ren, Jiadong
    Fu, Zhangjie
    Fu, Songling
    Hu, Yongjian
    Li, Chang-Tsun
    [J]. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2018, 86 : 995 - 1007
  • [50] Rapid modelling of the large-scale magnetospheric field from Swarm satellite data
    Hamilton, Brian
    [J]. EARTH PLANETS AND SPACE, 2013, 65 (11): : 1295 - 1308