A Study on Load Balancing Techniques for Task Allocation in Big Data Processing

被引：0
|
作者：
Jin Xiaohong [1 ]
Li Hui [1 ]
Liu Yanjun [1 ]
Fan Yanfang [1 ]
机构：
[1] Beijing Inst Sci & Technol Informat, Beijing 100048, Peoples R China
来源：
PROCEEDINGS OF THE 2016 INTERNATIONAL FORUM ON MECHANICAL, CONTROL AND AUTOMATION (IFMCA 2016) | 2017年 / 113卷
关键词：
Big Data; Job Schedule; Distributed Computing; Clustering; Load Balancing;
D O I：
暂无
中图分类号：
TP39 [计算机的应用];
学科分类号：
081203 ; 0835 ;
摘要：
This paper introduces the task allocation techniques with clustering and load balancing in the field of Internet to the field of image processing job allocation of alternative big data. It designs and realizes a load balancing cluster architecture for the alternative big data, and an improved load balancing algorithm applicable to large-scale image processing. The experimental results show that the cluster architecture can execute task allocation and data processing continuously and stably, and the improved load balancing algorithm could improve the processing efficiency about 10% and more.
引用
收藏
页码：212 / 218
页数：7
相关论文
共 50 条
  • [21] Load Balancing in Control Allocation
    Bodson, Marc
    Frost, Susan A.
    JOURNAL OF GUIDANCE CONTROL AND DYNAMICS, 2011, 34 (02) : 380 - 387
  • [22] Research on dynamic load balancing of data flow under big data platform
    Sun, Junlin
    Zhang, Yi
    INTERNATIONAL JOURNAL OF MODELING SIMULATION AND SCIENTIFIC COMPUTING, 2021, 12 (02)
  • [23] A distributed load balancing algorithm for climate big data processing over a multi-core CPU cluster
    Wang, Yuzhu
    Jiang, Jinrong
    Ye, Huang
    He, Juanxiong
    CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2016, 28 (15): : 4144 - 4160
  • [24] Load Balancing Scheme for Supporting Real-time Processing of Big Data in Distributed In-Memory Systems
    Bok, Kyoungsoo
    Choi, Kitae
    Lim, Jongtae
    Yoo, Jaesoo
    PROCEEDINGS OF THE 2018 CONFERENCE ON RESEARCH IN ADAPTIVE AND CONVERGENT SYSTEMS (RACS 2018), 2018, : 170 - 174
  • [25] An efficient and scalable SPARQL query processing framework for big data using MapReduce and hybrid optimum load balancing
    Kumar, V. Naveen
    Kumar, P. S. Ashok
    DATA & KNOWLEDGE ENGINEERING, 2023, 148
  • [26] Load balancing in task farms
    Wagner, AS
    Parsa, M
    INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED PROCESSING TECHNIQUES AND APPLICATIONS, VOLS I-III, PROCEEDINGS, 1997, : 947 - 956
  • [27] An Empirical Assessment of Performance of Data Balancing Techniques in Classification Task
    Jadhav, Anil
    Mostafa, Samih M.
    Elmannai, Hela
    Karim, Faten Khalid
    APPLIED SCIENCES-BASEL, 2022, 12 (08):
  • [28] Big Data Processing at the Edge with Data Skew Aware Resource Allocation
    Ahmadvand, Hossein
    Dargahi, Tooska
    Foroutan, Fouzhan
    Okorie, Princewill
    Esposito, Flavio
    2021 IEEE CONFERENCE ON NETWORK FUNCTION VIRTUALIZATION AND SOFTWARE DEFINED NETWORKS (IEEE NFV-SDN), 2021, : 81 - 86
  • [29] Context-aware Task Allocation for Fast Parallel Big Data Processing in Optical-Wireless Networks
    Suto, Katsuya
    Nishiyama, Hiroki
    Kato, Nei
    2014 INTERNATIONAL WIRELESS COMMUNICATIONS AND MOBILE COMPUTING CONFERENCE (IWCMC), 2014, : 423 - 428
  • [30] Dynamic Load Balancing Techniques for Distributed Complex Event Processing Systems
    Zacheilas, Nikos
    Zygouras, Nikolas
    Panagiotou, Nikolaos
    Kalogeraki, Vana
    Gunopulos, Dimitrios
    DISTRIBUTED APPLICATIONS AND INTEROPERABLE SYSTEMS, DAIS 2016, 2016, 9687 : 174 - 188
12345