Sharing lattice QCD data over a widely distributed file system

被引:19
|
作者
Amagasa, T. [1 ]
Aoki, S. [2 ]
Aoki, Y. [3 ]
Aoyama, T. [3 ]
Doi, T. [4 ]
Fukumura, K. [2 ]
Ishii, N. [5 ]
Ishikawa, K. -I [6 ]
Jitsumoto, H. [7 ]
Kamano, H. [5 ]
Konno, Y. [11 ]
Matsufuru, H. [8 ]
Mikami, Y. [11 ]
Miura, K. [3 ]
Sato, M. [1 ]
Takeda, S. [9 ]
Tatebe, O. [1 ]
Togawa, H. [5 ]
Ukawa, A. [10 ]
Ukita, N. [1 ]
Watanabe, Y. [4 ]
Yamazaki, T. [1 ]
Yoshie, T. [1 ]
机构
[1] Univ Tsukuba, Ctr Computat Sci, Tsukuba, Ibaraki 3058577, Japan
[2] Kyoto Univ, Yukawa Inst Theoret Phys, Kyoto, Kyoto 6068502, Japan
[3] Nagoya Univ, Kobayashi Maskawa Inst Origin Particles & Univers, Nagoya, Aichi 4648602, Japan
[4] RIKEN Nishina Ctr, Wako, Saitama 3510198, Japan
[5] Osaka Univ, Nucl Phys Res Ctr, Ibaraki, Osaka 5670047, Japan
[6] Hiroshima Univ, Grad Sch Sci, Hiroshima 7398526, Japan
[7] Univ Tokyo, Ctr Informat Technol, Bunkyo Ku, Tokyo 1138658, Japan
[8] KEK, Ctr Res Comp, Tsukuba, Ibaraki 3050801, Japan
[9] Kanazawa Univ, Inst Phys, Kanazawa, Ishikawa 9201192, Japan
[10] RIKEN Adv Inst Computat Sci, Kobe, Hyogo 6500047, Japan
[11] Hitachi Solut East Japan Ltd, Kawasaki, Kanagawa 2100007, Japan
关键词
D O I
10.1088/1742-6596/664/4/042058
中图分类号
O57 [原子核物理学、高能物理学];
学科分类号
070202 ;
摘要
JLDG is a data-grid for the lattice QCD (LQCD) community in Japan. Several large research groups in Japan have been working on lattice QCD simulations using supercomputers distributed over distant sites. The JLDG provides such collaborations with an efficient method of data management and sharing. File servers installed on 9 sites are connected to the NH SINET VPN and are bound into a single file system with the GFarm. The file system looks the same from any sites, so that users can do analyses on a supercomputer on a site, using data generated and stored in the JLDG at a different site. We present a brief description of hardware and software of the JLDG, including a recently developed subsystem for cooperating with the HPCI shared storage, and report performance and statistics of the JLDG. As of April 2015, 15 research groups (61 users) store their daily research data of 4.7PB including replica and 68 million files in total. Number of publications for works which used the JLDG is 98. The large number of publications and recent rapid increase of disk usage convince us that the JLDG has grown up into a useful infrastructure for LQCD community in Japan.
引用
收藏
页数:8
相关论文
共 50 条
  • [31] A self-encryption based private storage system over P2P distributed file sharing infrastructure
    Endo, Hiroki
    Kawahara, Yoshihiro
    Asami, Tohru
    PROCEEDINGS OF THE FIRST ITU-T KALEIDOSCOPE ACADEMIC CONFERENCE INNOVATIONS IN NGN: FUTURE NETWORK AND SERVICES, 2008, : 69 - 76
  • [32] Heterogeneous File Sharing by File-System Emulation
    Ing, Cherng-Ying
    Lee, Tzao-Lin
    2008 13TH ASIA-PACIFIC COMPUTER SYSTEMS ARCHITECTURE CONFERENCE, 2008, : 376 - 379
  • [33] A Secure Data Sharing Platform Using Blockchain and Interplanetary File System
    Naz, Muqaddas
    Al-zahrani, Fahad A.
    Khalid, Rabiya
    Javaid, Nadeem
    Qamar, Ali Mustafa
    Afzal, Muhammad Khalil
    Shafiq, Muhammad
    SUSTAINABILITY, 2019, 11 (24)
  • [34] File system security: Secure network data sharing for NT and UNIX
    Allison, B
    Hawley, R
    Borr, A
    Muhlestein, M
    Hitz, D
    PROCEEDINGS OF THE LARGE INSTALLATION SYSTEM ADMINISTRATION OF WINDOWS NT CONFERENCE, 1998, : 17 - 26
  • [35] Distributed Data Management and Distributed File Systems
    Girone, Maria
    21ST INTERNATIONAL CONFERENCE ON COMPUTING IN HIGH ENERGY AND NUCLEAR PHYSICS (CHEP2015), PARTS 1-9, 2015, 664
  • [36] On File Sharing Over a Wireless Social Network
    Chen, Yi-Ting
    Caramanis, Constantine
    Shakkottai, Sanjay
    2011 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY PROCEEDINGS (ISIT), 2011,
  • [37] A secure file sharing service for distributed computing environments
    Aniello Castiglione
    Luigi Catuogno
    Aniello Del Sorbo
    Ugo Fiore
    Francesco Palmieri
    The Journal of Supercomputing, 2014, 67 : 691 - 710
  • [38] A secure file sharing service for distributed computing environments
    Castiglione, Aniello
    Catuogno, Luigi
    Del Sorbo, Aniello
    Fiore, Ugo
    Palmieri, Francesco
    JOURNAL OF SUPERCOMPUTING, 2014, 67 (03): : 691 - 710
  • [39] Data acquisition and sharing for a distributed intelligent DSM system
    Andria, Gregorio
    Di Lecce, Vincenzo
    2007 IEEE INSTRUMENTATION & MEASUREMENT TECHNOLOGY CONFERENCE, VOLS 1-5, 2007, : 1242 - +
  • [40] On a secure distributed data sharing system and its implementation
    Kasza, Peter
    Ligeti, Peter
    Nagy, Adam
    ANNALES MATHEMATICAE ET INFORMATICAE, 2015, 44 : 111 - 120