Sharing lattice QCD data over a widely distributed file system

被引：19

作者：

Amagasa, T. ^{[1
]}

Aoki, S. ^{[2
]}

Aoki, Y. ^{[3
]}

Aoyama, T. ^{[3
]}

Doi, T. ^{[4
]}

Fukumura, K. ^{[2
]}

Ishii, N. ^{[5
]}

Ishikawa, K. -I ^{[6
]}

Jitsumoto, H. ^{[7
]}

Kamano, H. ^{[5
]}

Konno, Y. ^{[11
]}

Matsufuru, H. ^{[8
]}

Mikami, Y. ^{[11
]}

Miura, K. ^{[3
]}

Sato, M. ^{[1
]}

Takeda, S. ^{[9
]}

Tatebe, O. ^{[1
]}

Togawa, H. ^{[5
]}

Ukawa, A. ^{[10
]}

Ukita, N. ^{[1
]}

Watanabe, Y. ^{[4
]}

Yamazaki, T. ^{[1
]}

Yoshie, T. ^{[1
]}

机构：

[1] Univ Tsukuba, Ctr Computat Sci, Tsukuba, Ibaraki 3058577, Japan

[2] Kyoto Univ, Yukawa Inst Theoret Phys, Kyoto, Kyoto 6068502, Japan

[3] Nagoya Univ, Kobayashi Maskawa Inst Origin Particles & Univers, Nagoya, Aichi 4648602, Japan

[4] RIKEN Nishina Ctr, Wako, Saitama 3510198, Japan

[5] Osaka Univ, Nucl Phys Res Ctr, Ibaraki, Osaka 5670047, Japan

[6] Hiroshima Univ, Grad Sch Sci, Hiroshima 7398526, Japan

[7] Univ Tokyo, Ctr Informat Technol, Bunkyo Ku, Tokyo 1138658, Japan

[8] KEK, Ctr Res Comp, Tsukuba, Ibaraki 3050801, Japan

[9] Kanazawa Univ, Inst Phys, Kanazawa, Ishikawa 9201192, Japan

[10] RIKEN Adv Inst Computat Sci, Kobe, Hyogo 6500047, Japan

[11] Hitachi Solut East Japan Ltd, Kawasaki, Kanagawa 2100007, Japan

来源：

21ST INTERNATIONAL CONFERENCE ON COMPUTING IN HIGH ENERGY AND NUCLEAR PHYSICS (CHEP2015), PARTS 1-9 | 2015年 / 664卷

关键词：

D O I：

10.1088/1742-6596/664/4/042058

中图分类号：

O57 [原子核物理学、高能物理学];

学科分类号：

070202 ;

摘要：

JLDG is a data-grid for the lattice QCD (LQCD) community in Japan. Several large research groups in Japan have been working on lattice QCD simulations using supercomputers distributed over distant sites. The JLDG provides such collaborations with an efficient method of data management and sharing. File servers installed on 9 sites are connected to the NH SINET VPN and are bound into a single file system with the GFarm. The file system looks the same from any sites, so that users can do analyses on a supercomputer on a site, using data generated and stored in the JLDG at a different site. We present a brief description of hardware and software of the JLDG, including a recently developed subsystem for cooperating with the HPCI shared storage, and report performance and statistics of the JLDG. As of April 2015, 15 research groups (61 users) store their daily research data of 4.7PB including replica and 68 million files in total. Number of publications for works which used the JLDG is 98. The large number of publications and recent rapid increase of disk usage convince us that the JLDG has grown up into a useful infrastructure for LQCD community in Japan.

引用

页数：8

共 50 条

[41] A home-grown but widely-distributed data analysis system
Liszt, HS
ASTRONOMICAL DATA ANALYSIS SOFTWARE AND SYSTEMS VI, 1997, 125 : 3 - 9
[42] THE PULSE DISTRIBUTED FILE SYSTEM
TOMLINSON, GM
KEEFFE, D
WAND, IC
WELLINGS, AJ
SOFTWARE-PRACTICE & EXPERIENCE, 1985, 15 (11): : 1087 - 1101
[43] Distributed hypercube file system
Flynn, R.J.
Hadimioglu, H.
Conference on Hypercube Concurrent Computers and Applications, 1988,
[44] A DISTRIBUTED FILE SYSTEM FOR MINIX
GAMMILL, R
HERNES, J
SCCS PROCEEDINGS : 22ND ANNUAL SMALL COLLEGE COMPUTING SYMPOSIUM, 1989, : 151 - 160
[45] The Hadoop Distributed File System
Shvachko, Konstantin
Kuang, Hairong
Radia, Sanjay
Chansler, Robert
2010 IEEE 26TH SYMPOSIUM ON MASS STORAGE SYSTEMS AND TECHNOLOGIES (MSST), 2010,
[46] DISTRIBUTED MULTIMEDIA FILE SYSTEM
Radovici, Alexandru
Miu, Tudor
Dragoi, George
ANNALS OF DAAAM FOR 2009 & PROCEEDINGS OF THE 20TH INTERNATIONAL DAAAM SYMPOSIUM, 2009, 20 : 593 - 594
[47] Scalable secure file sharing system
NTT Information Sharing Platform Laboratories, Musashino-shi, 180-8585, Japan
不详
不详
NTT Tech. Rev., 2006, 10 (60-64):
[48] ALDM: Algorithm of Adaptive Loading Data Migration in Distributed File System
Tan Zhipeng
Zhou Wei
Zhang Wenhua
Feng Dan
2012 DIGEST ASIA-PACIFIC MAGNETIC RECORDING CONFERENCE (APMRC), 2012,
[49] Computer Performance Determination System Based on Big Data Distributed File
Lu, Kong
CYBER SECURITY INTELLIGENCE AND ANALYTICS, 2020, 928 : 877 - 884
[50] Analysis of DNA Data Using Hadoop Distributed File System.
Senthilkumar, M.
Ilango, P.
RESEARCH JOURNAL OF PHARMACEUTICAL BIOLOGICAL AND CHEMICAL SCIENCES, 2016, 7 (03): : 796 - 803

← 1 2 3 4 5 →