Optimization of Large Scale of Files Transfer in Meteorological Grid Optimization of Large Scale of Files Transfer in Meteorological Grid

被引：0

作者：

Ma, Tinghuai ^{[1
]}

Cao, Hao ^{[2
]}

Wang, Jin ^{[2
]}

Tian, Wei ^{[2
]}

Dickson, Keddy Wornyo ^{[2
]}

机构：

[1] Nanjing Univ Informat Sci & Technol, Jiangsu Engn Ctr Network Monitoring, Nanjing, Jiangsu, Peoples R China

[2] Nanjing Univ Informat Sci & Technol, Sch Comp & Software, Nanjing, Jiangsu, Peoples R China

来源：

INTERNATIONAL JOURNAL OF GRID AND DISTRIBUTED COMPUTING | 2013年 / 6卷 / 01期

基金：

美国国家科学基金会;

关键词：

Small Files; transfer performance; GridFTP; Compression; Meteorological Grid;

D O I：

暂无

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

This paper focuses on performance enhancement of large scale of small files transfer, which is critical to the performance of meteorological grid. GridFTP and compression techniques are used to optimize the efficiency. The transfer parameters are configured before transmission, such as extended block mode (Mode E), TCP buffer size. Compression is used to compress files into single file, in order to eliminate negotiating time before transmission. During the course of our research, we sacrifice the disk space for avoiding the vicious recurrence of compression-decompression-compression. In the end, the experiments show the advantages of our approach.

引用

页码：25 / 33

页数：9

共 50 条

[1] Large-Scale Grid Optimization: the Workhorse of Future Grid Computations
Pandey A.
Almassalkhi M.R.
Chevalier S.
[J]. Current Sustainable/Renewable Energy Reports, 2023, 10 (03): : 139 - 153
[2] Constructing a Web service system for large-scale meteorological grid data
Amagasa, Toshiyuki
Kitagawa, Hiroyuki
Komano, Tatsuya
[J]. E-SCIENCE 2007: THIRD IEEE INTERNATIONAL CONFERENCE ON E-SCIENCE AND GRID COMPUTING, PROCEEDINGS, 2007, : 118 - +
[3] Global, data transfer of large files
Fairbairn, Scott
Koehler, Steffen
Ogden, Anthony
[J]. R&D MAGAZINE, 2007, 49 (11): : 46 - 47
[4] Parameter optimization in 3D reconstruction on a large scale grid
Bilbao-Castro, J. R.
Merino, A.
Garcia, I.
Carazo, J. M.
Fernandez, J. J.
[J]. PARALLEL COMPUTING, 2007, 33 (4-5) : 250 - 263
[5] Resource Discovery and Selection for Large Scale Query Optimization in a Grid Environment
El Samad, Mahmoud
Hameurlain, Abdelkader
Morvan, Franck
[J]. ADVANCES IN COMPUTER AND INFORMATIOM SCIENCES AND ENGINEERING, 2008, : 279 - 283
[6] Solving large scale optimization problems via grid and cluster computing
Fujisawa, K
Kojima, M
Takeda, A
Yamashita, M
[J]. JOURNAL OF THE OPERATIONS RESEARCH SOCIETY OF JAPAN, 2004, 47 (04) : 265 - 274
[7] Performance and Cost Optimization for Multiple Large-scale Grid Workflow Applications
Duan, Rubing
Prodan, Radu
Fahringer, Thomas
[J]. 2007 ACM/IEEE SC07 CONFERENCE, 2010, : 500 - 511
[8] Energy Management and Optimization of Large-Scale Electric Vehicle Charging on the Grid
Kene, Raymond O.
Olwal, Thomas O.
[J]. WORLD ELECTRIC VEHICLE JOURNAL, 2023, 14 (04):
[9] Large-Scale Identification of Malicious Singleton Files
Li, Bo
Roundy, Kevin
Gates, Chris
Vorobeychik, Yevgeniy
[J]. PROCEEDINGS OF THE SEVENTH ACM CONFERENCE ON DATA AND APPLICATION SECURITY AND PRIVACY (CODASPY'17), 2017, : 227 - 238
[10] A study on the optimization of emergency transfer of casualties in large-scale disasters
Zhang Mingxing
Cheng Guangquan
Zhu Xianqiang
Liu Zhong
[J]. 2018 EIGHTH INTERNATIONAL CONFERENCE ON INSTRUMENTATION AND MEASUREMENT, COMPUTER, COMMUNICATION AND CONTROL (IMCCC 2018), 2018, : 1711 - 1716

← 1 2 3 4 5 →