IDP: An Innovative Data Placement Algorithm for Hadoop Systems

被引：2
|
作者：
Lee, Chia-Wei [1 ]
Huang, Horng-Chyau [1 ]
Hsieh, Sun-Yuan [1 ,2 ,3 ]
机构：
[1] Natl Cheng Kung Univ, Inst Med Informat, 1 Univ Rd, Tainan 701, Taiwan
[2] Natl Cheng Kung Univ, Dept Comp Sci & Informat Engn, Tainan 701, Taiwan
[3] Natl Cheng Kung Univ, Inst Mfg Informat & Syst, Tainan 701, Taiwan
来源：
INTELLIGENT SYSTEMS AND APPLICATIONS (ICS 2014) | 2015年 / 274卷
关键词：
Data Placement; Hadoop; Heterogeneous; MapReduce;
D O I：
10.3233/978-1-61499-484-8-49
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
In this paper, we propose a data placement strategy to deal with the imbalanced workload problem on DataNodes. Basing on computing capability of each node in a heterogeneous Hadoop cluster, the proposed strategy can balance the data that was stored in the DataNode such that the cost of data transfer time can be tremendously reduced. As a result, the Hadoop overall performance can be greatly improved. Experimental results demonstrate that the proposed data placement strategy can highly decrease the execution time and thus improves Hadoop performance in a heterogeneous cluster.
引用
收藏
页码：49 / 58
页数：10
相关论文
共 50 条
  • [1] Enhanced Bond Energy Algorithm for Data Placement in Hadoop Framework
    Sridevi, S.
    Reshma, J. G.
    Pavithradevi, E.
    Dhivya, S.
    Uthariaraj, V. Rhymend
    2018 10TH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTING (ICOAC), 2018, : 208 - 215
  • [2] A Cost-Efficient Data Placement Algorithm with High Reliability in Hadoop
    Du, Yao
    Xiong, Runqun
    Jin, Jiahui
    Luo, Junzhou
    2017 FIFTH INTERNATIONAL CONFERENCE ON ADVANCED CLOUD AND BIG DATA (CBD), 2017, : 100 - 105
  • [3] An improved data placement strategy for hadoop
    Lin, Wei-Wei
    Huanan Ligong Daxue Xuebao/Journal of South China University of Technology (Natural Science), 2012, 40 (01): : 152 - 158
  • [4] New Data Placement Strategy in the HADOOP Framework
    Elomari, Akram
    Hassouni, Larbi
    Maizate, Abderrahim
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2021, 12 (07) : 676 - 684
  • [5] Optimizing data placement in heterogeneous Hadoop clusters
    Runqun Xiong
    Junzhou Luo
    Fang Dong
    Cluster Computing, 2015, 18 : 1465 - 1480
  • [6] Optimizing data placement in heterogeneous Hadoop clusters
    Xiong, Runqun
    Luo, Junzhou
    Dong, Fang
    CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2015, 18 (04): : 1465 - 1480
  • [7] An improved data placement strategy in a heterogeneous Hadoop cluster
    Zhao, Wentao
    Meng, Lingjun
    Sun, Jiangfeng
    Ding, Yang
    Zhao, Haohao
    Wang, Lina
    Open Cybernetics and Systemics Journal, 2014, 8 (01): : 957 - 963
  • [8] On a Dynamic Data Placement Strategy for Heterogeneous Hadoop Clusters
    Liu, Yang
    Wu, Chase Q.
    Wang, Meng
    Hou, Aiqin
    Wang, Yongqiang
    2018 INTERNATIONAL SYMPOSIUM ON NETWORKS, COMPUTERS AND COMMUNICATIONS (ISNCC 2018), 2018,
  • [9] CoHadoop: Flexible Data Placement and Its Exploitation in Hadoop
    Eltabakh, Mohamed Y.
    Tian, Yuanyuan
    Ozcan, Fatma
    Gemulla, Rainer
    Krettek, Aljoscha
    McPherson, John
    PROCEEDINGS OF THE VLDB ENDOWMENT, 2011, 4 (09): : 575 - 585
  • [10] An Improved data placement strategy in a heterogeneous hadoop cluster
    Zhao, Wentao
    Meng, Lingjun
    Sun, Jiangfeng
    Ding, Yang
    Zhao, Haohao
    Wang, Lina
    Open Cybernetics and Systemics Journal, 2015, 9 (01): : 792 - 798
12345