Modelling Large-Scale Scientific Data Transfers

被引：0
|
作者：
Bogado J. [1 ,2 ]
Lassnig M. [3 ]
Monticelli F. [2 ]
Díaz J. [1 ]
机构：
[1] LINTI, Facultad de Informática, La Plata
[2] IFLP, UNLP, CONICET, La Plata
[3] European Organization for Nuclear Research (CERN), Geneva
来源：
Computing and Software for Big Science | 2022年 / 6卷 / 1期
关键词：
Data transfer analysis; Distributed computing modelling; Performance metrics;
D O I：
10.1007/s41781-022-00084-4
中图分类号：
学科分类号：
摘要：
This work focuses on the study of a recently published dataset (Bogado et al. in ATLAS Rucio transfers dataset. Zenodo, 2020.) with data that allow us to reconstruct the lifetime of file transfers in the contexts of the Worldwide LHC Computing Grid (WLCG). Several models for Rule Time To Complete (TTC) prediction are presented and evaluated. The dataset source is Rucio, an open-source software framework that provides scientific collaborations with the functionality to organize, manage, and access their data at scale. The rich amount of data gathered about the transfers and rules, presents a unique opportunity to better understand the complex mechanisms involved in file transfers across the WLCG. © 2022, The Author(s).
引用
收藏
相关论文
共 50 条
  • [31] A Dynamic Programmable Network for Large-Scale Scientific Data Transfer Using AmoebaNet
    Shah, Syed Asif Raza
    Noh, Seo-Young
    [J]. APPLIED SCIENCES-BASEL, 2019, 9 (21):
  • [32] On a Pipeline-based Architecture for Parallel Visualization of Large-scale Scientific Data
    Chu, Dongliang
    Wu, Chase Q.
    [J]. PROCEEDINGS OF 45TH INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING WORKSHOPS (ICPPW 2016), 2016, : 88 - 97
  • [33] EventDB: A Large-Scale Semi-structured Scientific Data Management System
    Zhao, Wenjia
    Qi, Yong
    Hou, Di
    Wang, Peijian
    Gao, Xin
    Du, Zirong
    Zhang, Yudong
    Zong, Yongfang
    [J]. BIG SCIENTIFIC DATA MANAGEMENT, 2019, 11473 : 105 - 115
  • [34] Big data techniques: Large-scale text analysis for scientific and journalistic research
    Arcila-Calderon, Carlos
    Barbosa-Caro, Eduar
    Cabezuelo-Lorenzo, Francisco
    [J]. PROFESIONAL DE LA INFORMACION, 2016, 25 (04): : 623 - 631
  • [35] SciAP: A Programmable, High-Performance Platform for Large-Scale Scientific Data
    Tian, Yang
    Li, Chao
    Liu, Chao
    Yan, Haihua
    [J]. 2018 INTERNATIONAL CONFERENCE ON CLOUD COMPUTING, BIG DATA AND BLOCKCHAIN (ICCBB 2018), 2018, : 148 - 154
  • [36] An architecture and implementation to support large-scale data access in scientific simulation environments
    Holmes, VP
    Kleban, SD
    Miller, DJ
    Pavlakos, C
    Poore, CA
    Vandewart, RL
    Crowley, CP
    [J]. 35TH ANNUAL SIMULATION SYMPOSIUM, PROCEEDINGS, 2002, : 169 - 176
  • [37] A high-performance application data environment for large-scale scientific computations
    Shen, XH
    Liao, WK
    Chouldhary, A
    Memik, G
    Kandemir, M
    [J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2003, 14 (12) : 1262 - 1274
  • [38] Named Data Networking Strategies for Improving Large Scientific Data Transfers
    Shannigrahi, Susmit
    Fan, Chengyu
    Papadopoulos, Christos
    [J]. 2018 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS WORKSHOPS (ICC WORKSHOPS), 2018,
  • [39] Large-scale data visualization
    Ma, KL
    [J]. IEEE COMPUTER GRAPHICS AND APPLICATIONS, 2001, 21 (04) : 22 - 23
  • [40] Securing Large-scale Data Transfers in Campus Networks: Experiences, Issues, and Challenges (Invited Paper)
    Nadig, Deepak
    Ramamurthy, Byrav
    [J]. PROCEEDINGS OF THE ACM INTERNATIONAL WORKSHOP ON SECURITY IN SOFTWARE DEFINED NETWORKS & NETWORK FUNCTION VIRTUALIZATION (SDN-NFV '19), 2019, : 29 - 32
12345