A Study on Load Balancing Techniques for Task Allocation in Big Data Processing

被引：0
|
作者：
Jin Xiaohong [1 ]
Li Hui [1 ]
Liu Yanjun [1 ]
Fan Yanfang [1 ]
机构：
[1] Beijing Inst Sci & Technol Informat, Beijing 100048, Peoples R China
来源：
PROCEEDINGS OF THE 2016 INTERNATIONAL FORUM ON MECHANICAL, CONTROL AND AUTOMATION (IFMCA 2016) | 2017年 / 113卷
关键词：
Big Data; Job Schedule; Distributed Computing; Clustering; Load Balancing;
D O I：
暂无
中图分类号：
TP39 [计算机的应用];
学科分类号：
081203 ; 0835 ;
摘要：
This paper introduces the task allocation techniques with clustering and load balancing in the field of Internet to the field of image processing job allocation of alternative big data. It designs and realizes a load balancing cluster architecture for the alternative big data, and an improved load balancing algorithm applicable to large-scale image processing. The experimental results show that the cluster architecture can execute task allocation and data processing continuously and stably, and the improved load balancing algorithm could improve the processing efficiency about 10% and more.
引用
收藏
页码：212 / 218
页数：7
相关论文
共 50 条
  • [31] Load balancing techniques for distributed stream processing applications in overlay environments
    Drougas, Yannis
    Repantis, Thomas
    Kalogeraki, Vana
    NINTH IEEE INTERNATIONAL SYMPOSIUM ON OBJECT AND COMPONENT-ORIENTED REAL-TIME DISTRIBUTED COMPUTING, PROCEEDINGS, 2006, : 33 - 40
  • [32] A new load balancing strategy by task allocation in edge computing based on intermediary nodes
    Li, Guangshun
    Yao, Yonghui
    Wu, Junhua
    Liu, Xiaoxiao
    Sheng, Xiaofei
    Lin, Qingyan
    EURASIP JOURNAL ON WIRELESS COMMUNICATIONS AND NETWORKING, 2020, 2020 (01)
  • [33] A new load balancing strategy by task allocation in edge computing based on intermediary nodes
    Guangshun Li
    Yonghui Yao
    Junhua Wu
    Xiaoxiao Liu
    Xiaofei Sheng
    Qingyan Lin
    EURASIP Journal on Wireless Communications and Networking, 2020
  • [34] A generic API for load balancing in distributed systems for big data management
    Antoine, Maeva
    Pellegrino, Laurent
    Huet, Fabrice
    Baude, Francoise
    CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2016, 28 (08): : 2440 - 2456
  • [35] Load Balancing for Privacy-Preserving Access to Big Data in Cloud
    Li, Peng
    Guo, Song
    2014 IEEE CONFERENCE ON COMPUTER COMMUNICATIONS WORKSHOPS (INFOCOM WKSHPS), 2014, : 524 - 528
  • [36] Data allocation and dynamic load balancing for distributed video storage server
    J Visual Commun Image Represent, 2 (197-218):
  • [37] Dynamic Optical Data Center Network Load Balancing and Resource Allocation
    Huang, Henna
    Chan, Vincent W. S.
    2016 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM), 2016,
  • [38] Data allocation and dynamic load balancing for distributed video storage server
    Tsao, SL
    Chen, MC
    Ko, MT
    Ho, JM
    Huang, YM
    JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 1999, 10 (02) : 197 - 218
  • [39] A systematic literature review for load balancing and task scheduling techniques in cloud computing
    Devi, Nisha
    Dalal, Sandeep
    Solanki, Kamna
    Dalal, Surjeet
    Lilhore, Umesh Kumar
    Simaiya, Sarita
    Nuristani, Nasratullah
    ARTIFICIAL INTELLIGENCE REVIEW, 2024, 57 (10)
  • [40] A Load Balancing Framework In Cloud To Support Task Processing Using Smartphone With CWC
    Salame, Anantkumar Vikas
    Kulkarni, Pradnya V.
    2018 3RD INTERNATIONAL CONFERENCE FOR CONVERGENCE IN TECHNOLOGY (I2CT), 2018,
12345