Rucio: Scientific Data Management

被引:79
|
作者
Barisits M. [1 ]
Beermann T. [2 ]
Berghaus F. [3 ]
Bockelman B. [4 ]
Bogado J. [5 ]
Cameron D. [6 ]
Christidis D. [7 ]
Ciangottini D. [8 ]
Dimitrov G. [1 ]
Elsing M. [1 ]
Garonne V. [6 ]
di Girolamo A. [1 ]
Goossens L. [1 ]
Guan W. [9 ]
Guenther J. [1 ]
Javurek T. [1 ]
Kuhn D. [10 ]
Lassnig M. [1 ]
Lopez F. [5 ]
Magini N. [11 ]
Molfetas A. [1 ]
Nairz A. [1 ]
Ould-Saada F. [6 ]
Prenner S. [10 ]
Serfon C. [10 ]
Stewart G. [1 ]
Vaandering E. [12 ]
Vasileva P. [1 ]
Vigne R. [13 ]
Wegner T. [1 ]
机构
[1] CERN, Meyrin
[2] University of Wuppertal, Wuppertal
[3] University of Victoria, Victoria
[4] Morgridge Institute, Madison, WI
[5] Universidad Nacional de La Plata, La Plata
[6] University of Oslo, Oslo
[7] University of Texas at Arlington, Arlington, TX
[8] INFN, Perugia
[9] University of Wisconsin-Madison, Madison, WI
[10] University of Innsbruck, Innsbruck
[11] Iowa State University, Ames, IA
[12] Fermi National Accelerator Laboratory, Batavia, IL
[13] University of Vienna, Vienna
关键词
Data access; Data management; Data organization; Distributed computing; Exascale;
D O I
10.1007/s41781-019-0026-3
中图分类号
学科分类号
摘要
Rucio is an open-source software framework that provides scientific collaborations with the functionality to organize, manage, and access their data at scale. The data can be distributed across heterogeneous data centers at widely distributed locations. Rucio was originally developed to meet the requirements of the high-energy physics experiment ATLAS, and now is continuously extended to support the LHC experiments and other diverse scientific communities. In this article, we detail the fundamental concepts of Rucio, describe the architecture along with implementation details, and report operational experience from production usage. © 2019, The Author(s).
引用
收藏
相关论文
共 50 条
  • [11] Records management for scientific data
    Maday, Charlotte
    Moysan, Magalie
    ARCHIVES AND MANUSCRIPTS, 2014, 42 (02) : 190 - 192
  • [12] Scientific Data Management Software
    不详
    CHIMICA OGGI-CHEMISTRY TODAY, 2002, 20 (05) : 62 - 62
  • [13] Evolution of the open-source data management system Rucio for LHC Run-3 and beyond ATLAS
    Barisits, Martin
    Beermann, Thomas
    Bogado, Joaquin
    Garonne, Vincent
    Javurek, Tomas
    Lassnig, Mario
    Manzi, Andrea
    Martelli, Edoardo
    Serfon, Cedric
    Wegner, Tobias
    Zhao, Xin
    23RD INTERNATIONAL CONFERENCE ON COMPUTING IN HIGH ENERGY AND NUCLEAR PHYSICS (CHEP 2018), 2019, 214
  • [14] DaltOn:: An infrastructure for scientific data management
    Jablonski, Stefan
    Cure, Olivier
    Rehman, M. Abdul
    Volz, Bernhard
    COMPUTATIONAL SCIENCE - ICCS 2008, PT 3, 2008, 5103 : 520 - +
  • [15] Sequence data management for scientific purposes
    Margos, Gabriele
    Fingerle, Volker
    Sing, Andreas
    Jolley, Keith
    INFECTION GENETICS AND EVOLUTION, 2017, 54 : 508 - 508
  • [16] Data Management Tools for Scientific Analytics
    Balazinska, Magdalena
    OMICS-A JOURNAL OF INTEGRATIVE BIOLOGY, 2011, 15 (04) : 227 - 230
  • [17] Data management systems for scientific applications
    Moore, RW
    ARCHITECTURE OF SCIENTIFIC SOFTWARE, 2001, 60 : 273 - 284
  • [18] Scientific data management in a Grid environment
    James H.A.
    Hawick K.A.
    Journal of Grid Computing, 2005, 3 (1-2) : 39 - 51
  • [19] DataFoundry: Information management for scientific data
    Critchlow, T
    Fidelis, K
    Ganesh, M
    Musick, R
    Slezak, T
    IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, 2000, 4 (01): : 52 - 57
  • [20] Energy Smart Management of Scientific Data
    Otoo, Ekow
    Rotem, Doron
    Tsao, Shih-Chiang
    SCIENTIFIC AND STATISTICAL DATABASE MANAGEMENT, PROCEEDINGS, 2009, 5566 : 92 - 109