Research of the Strategy to Improve the Hadoop Distributed File System Downloading Performance

被引：0
|
作者：
Ge Junwei [1 ]
Jiang Renxiang [1 ]
机构：
[1] Chongqing Univ Posts & Telecommun, Chongqing, Peoples R China
来源：
ELECTRONIC INFORMATION AND ELECTRICAL ENGINEERING | 2012年 / 19卷
关键词：
cloud computing; hadoop distributed file system; parallel download; load balancing;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
This paper introduced the process of client downloading files from the Hadoop distributed file system, in the process of downloading files were studied for the user experience is poor and possible load imbalance problem, proposed a parallel download programs, and in the process of downloading data blocks from multiple backup data storage Datallodes, used an improved rate of adaptive dynamic parallel downloading mechanism. Experimental results show that the improved download strategy greatly reduces the waiting time of the client downloading files from the Hadoop distributed file system. Parallel download can improve Hadoop distributed file system download efficiency and make the Datallode more balanced load.
引用
收藏
页码：244 / 247
页数：4
相关论文
共 50 条
  • [31] A CKAN Plugin for Data Harvesting to the Hadoop Distributed File System
    Scholz, Robert
    Tcholtchev, Nikolay
    Laemmel, Philipp
    Schieferdecker, Ina
    [J]. CLOSER: PROCEEDINGS OF THE 7TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND SERVICES SCIENCE, 2017, : 19 - 28
  • [32] On the Power of In-Network Caching in the Hadoop Distributed File System
    Newberry, Eric
    Zhang, Beichuan
    [J]. PROCEEDINGS OF THE 2019 CONFERENCE ON INFORMATION-CENTRIC NETWORKING (ICN '19), 2019, : 89 - 99
  • [33] Complete Data Deletion Based on Hadoop Distributed File System
    Wang, Fulin
    Wu, Shunxiang
    Cai, Jianhuai
    Zhao, Longze
    Liao, Zhendong
    Ming, Daodong
    [J]. PROCEEDINGS OF THE THIRD INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND APPLICATION ENGINEERING (CSAE2019), 2019,
  • [34] A New Replica Placement Policy for Hadoop Distributed File System
    Dai, Wei
    Ibrahim, Ibrahim
    Bassiouni, Mostafa
    [J]. 2016 IEEE 2ND INTERNATIONAL CONFERENCE ON BIG DATA SECURITY ON CLOUD (BIGDATASECURITY), IEEE INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE AND SMART COMPUTING (HPSC), AND IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT DATA AND SECURITY (IDS), 2016, : 262 - 267
  • [35] Modeling and Simulation of Hadoop Distributed File System in a Cluster of Workstations
    Aguilera-Mendoza, Longendri
    Llorente-Quesada, Monica T.
    [J]. MODEL AND DATA ENGINEERING, MEDI 2013, 2013, 8216 : 1 - 12
  • [36] A Load-Balancing Algorithm for Hadoop Distributed File System
    Lin, Chi-Yi
    Lin, Ying-Chen
    [J]. PROCEEDINGS 2015 18TH INTERNATIONAL CONFERENCE ON NETWORK-BASED INFORMATION SYSTEMS (NBIS 2015), 2015, : 173 - 179
  • [37] Towards a Better Replica Management for Hadoop Distributed File System
    Ciritoglu, Hilmi Egemen
    Saber, Takfarinas
    Buda, Teodora Sandra
    Murphy, John
    Thorpe, Christina
    [J]. 2018 IEEE INTERNATIONAL CONGRESS ON BIG DATA (IEEE BIGDATA CONGRESS), 2018, : 104 - 111
  • [38] Dealing with Small Files Problem in Hadoop Distributed File System
    Bende, Sachin
    Shedge, Ashree
    [J]. PROCEEDINGS OF INTERNATIONAL CONFERENCE ON COMMUNICATION, COMPUTING AND VIRTUALIZATION (ICCCV) 2016, 2016, 79 : 1001 - 1012
  • [39] LOAD REBALANCING FOR HADOOP DISTRIBUTED FILE SYSTEM USING DISTRIBUTED HASH TABLE
    Nithya, M.
    Maheshwari, N. Uma
    [J]. PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON INTELLIGENT SUSTAINABLE SYSTEMS (ICISS 2017), 2017, : 939 - 943
  • [40] A Distributed NameNode Cluster for a Highly-Available Hadoop Distributed File System
    Kim, Yonghwan
    Araragi, Tadashi
    Nakamura, Junya
    Masuzawa, Toshimitsu
    [J]. 2014 IEEE 33RD INTERNATIONAL SYMPOSIUM ON RELIABLE DISTRIBUTED SYSTEMS (SRDS), 2014, : 333 - 334
12345