A Study on Load Balancing Techniques for Task Allocation in Big Data Processing

被引：0
|
作者：
Jin Xiaohong [1 ]
Li Hui [1 ]
Liu Yanjun [1 ]
Fan Yanfang [1 ]
机构：
[1] Beijing Inst Sci & Technol Informat, Beijing 100048, Peoples R China
来源：
PROCEEDINGS OF THE 2016 INTERNATIONAL FORUM ON MECHANICAL, CONTROL AND AUTOMATION (IFMCA 2016) | 2017年 / 113卷
关键词：
Big Data; Job Schedule; Distributed Computing; Clustering; Load Balancing;
D O I：
暂无
中图分类号：
TP39 [计算机的应用];
学科分类号：
081203 ; 0835 ;
摘要：
This paper introduces the task allocation techniques with clustering and load balancing in the field of Internet to the field of image processing job allocation of alternative big data. It designs and realizes a load balancing cluster architecture for the alternative big data, and an improved load balancing algorithm applicable to large-scale image processing. The experimental results show that the cluster architecture can execute task allocation and data processing continuously and stably, and the improved load balancing algorithm could improve the processing efficiency about 10% and more.
引用
收藏
页码：212 / 218
页数：7
相关论文
共 50 条
  • [1] A Load Balancing in Task Allocation of a Multiagent System
    Sombattheera, Chattrakul
    PROCEEDINGS OF 2014 2ND INTERNATIONAL SYMPOSIUM ON COMPUTATIONAL AND BUSINESS INTELLIGENCE (ISCBI), 2014, : 116 - 120
  • [2] Reliable Task Allocation with Load Balancing in Multiplex Networks
    Jiang, Yichuan
    Zhou, Yifeng
    Li, Yunpeng
    ACM TRANSACTIONS ON AUTONOMOUS AND ADAPTIVE SYSTEMS, 2015, 10 (01)
  • [3] A Survey of Task Allocation and Load Balancing in Distributed Systems
    Jiang, Yichuan
    IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2016, 27 (02) : 585 - 599
  • [4] Task offloading, load balancing, and resource allocation in MEC networks
    Li, S. L.
    Du, J. B.
    Zhai, D. S.
    Chu, X. L.
    Yu, F. R.
    IET COMMUNICATIONS, 2020, 14 (09) : 1451 - 1458
  • [5] GWO Based Task Allocation for Load Balancing in Containerized Cloud
    Patel, Dimple
    Patra, Manoj Kumar
    Sahoo, Bibhudatta
    PROCEEDINGS OF THE 5TH INTERNATIONAL CONFERENCE ON INVENTIVE COMPUTATION TECHNOLOGIES (ICICT-2020), 2020, : 655 - 659
  • [6] Slack allocation techniques for intra-path load balancing
    Gopalan, Kartik
    Chiueh, Tzi-cker
    Lin, Yow-Jian
    JOURNAL OF HIGH SPEED NETWORKS, 2007, 16 (03) : 211 - 237
  • [7] Load-balancing data prefetching techniques
    Chi, CH
    Yuan, JL
    FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2001, 17 (06): : 733 - 744
  • [8] Optimal allocation of servers and processing time in a load balancing system
    Maddah, Bacel
    El-Taha, Muhammad
    Abou Tayeh, Roy
    COMPUTERS & OPERATIONS RESEARCH, 2010, 37 (12) : 2173 - 2181
  • [9] Load balancing task allocation scheme in a hard real time system
    Lect Notes Comput Sci, (640):
  • [10] Load balancing task allocation of collaborative workshops based on immune algorithm
    Yu, XiaoYi
    Sun, ShuDong
    GLOBAL DESIGN TO GAIN A COMPETITIVE EDGE: AN HOLISTIC AND COLLABORATIVE DESIGN APPROACH BASED ON COMPUTATIONAL TOOLS, 2008, : 729 - 742
12345