DYNAMIC SCHEDULING ALGORITHM FOR REDUCING START TIME IN HADOOP

被引:3
|
作者
Gunasekaran, S. [1 ]
SaiRamesh, L. [2 ]
Sabena, S. [3 ]
Selvakumar, K. [4 ]
Ganapathy, S. [5 ]
Kannan, A. [2 ]
机构
[1] Anna Univ Chennai, Chennai, Tamil Nadu, India
[2] Anna Univ, Chennai, Tamil Nadu, India
[3] Anna Univ, Reg Ctr, Tirunelveli, India
[4] VIT Univ Vellore, Vellore, Tamil Nadu, India
[5] VIT Univ, Chennai, Tamil Nadu, India
关键词
Big data; Mapreduce; Self Adaptive Scheduling; Dynamic Scheduling; Task scheduling; MAPREDUCE;
D O I
10.1145/2980258.2982115
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Map Reduce is a model associated with a programming and implementation method and is used for formulating on large datasets. The main challenge is scaling of start blocks and present implementations might end in a block of scale back tasks. In this work, In this work, a new start up model is proposed using temporal constraints and hence, the map task gives a massive output then the performance of Map Reduce reduces drastically. Through this analysis the map reduce planning mechanism is modified to reduce the waste resources in the system slot. This tends to an end within the scale back tasks waiting around the proposed model scale back the planning policy for reducing the waiting of scales back tasks and begin times within the Hadoop platform. It also decides the beginning time and purpose of every scale back task dynamically based on the context of each job, together with the task completion time and therefore the size of map as output. Thereafter, scale back completion time and system average latent period job completion time have been estimated. The experimental results illustrate that the scale back completion time has been decreased sharply due to the rise of the temporal rules and map reduce techniques.
引用
收藏
页数:4
相关论文
共 50 条
  • [21] New Scheduling Algorithm in Hadoop Based on Resource Aware
    Xu, Peng
    Wang, Hong
    Tian, Ming
    [J]. PRACTICAL APPLICATIONS OF INTELLIGENT SYSTEMS, ISKE 2013, 2014, 279 : 1011 - 1020
  • [22] Deadline scheduling algorithm for sustainable computing in Hadoop environment
    Varga, Mihai
    Petrescu-Nita, Alina
    Pop, Florin
    [J]. COMPUTERS & SECURITY, 2018, 76 : 354 - 366
  • [23] Load balancing task scheduling algorithm in Hadoop platform
    Cai Yandong
    Liu Yan
    Zhang Qinglei
    [J]. 2015 SEVENTH INTERNATIONAL CONFERENCE ON MEASURING TECHNOLOGY AND MECHATRONICS AUTOMATION (ICMTMA 2015), 2015, : 605 - 608
  • [24] A dynamic scheduling algorithm for real-time expert systems
    Campos, AM
    García, DF
    [J]. ADVANCES IN ARTIFICIAL INTELLIGENCE - IBERAMIA 2002, PROCEEDINGS, 2002, 2527 : 775 - 784
  • [25] Profit-oriented task scheduling algorithm in Hadoop cluster
    Chai, Xu-qing
    Dong, Yong-liang
    Li, Jun-fei
    [J]. EURASIP JOURNAL ON EMBEDDED SYSTEMS, 2016,
  • [26] Improving Scheduling Efficiency of Hadoop YARN Using AFSA Algorithm
    Gao Junlei
    Tang Tie
    Wu Gangshan
    [J]. 2017 15TH IEEE INTERNATIONAL SYMPOSIUM ON PARALLEL AND DISTRIBUTED PROCESSING WITH APPLICATIONS AND 2017 16TH IEEE INTERNATIONAL CONFERENCE ON UBIQUITOUS COMPUTING AND COMMUNICATIONS (ISPA/IUCC 2017), 2017, : 919 - 924
  • [27] Job Scheduling Optimization using BAT Algorithm in Hadoop Environment
    Raghav, R. S.
    Amudhavel, J.
    Dhavachelvan, P.
    [J]. BIOSCIENCE BIOTECHNOLOGY RESEARCH COMMUNICATIONS, 2018, 11 (01): : 134 - 139
  • [28] MUS: a novel deadline-constrained scheduling algorithm for Hadoop
    Teng, Fei
    Yang, Hao
    Li, Tianrui
    Magoules, Frederic
    Fan, Xiaoliang
    [J]. INTERNATIONAL JOURNAL OF COMPUTATIONAL SCIENCE AND ENGINEERING, 2015, 11 (04) : 360 - 367
  • [29] A Radar Task Scheduling Method Using Random Shifted Start Time with the EST Algorithm
    Qu, Zhen
    Ding, Zhen
    Moo, Peter
    [J]. 2019 IEEE RADAR CONFERENCE (RADARCONF), 2019,
  • [30] A Dynamic and Failure-Aware Task Scheduling Framework for Hadoop
    Soualhia, Mbarka
    Khomh, Foutse
    Tahar, Sofiene
    [J]. IEEE TRANSACTIONS ON CLOUD COMPUTING, 2020, 8 (02) : 553 - 569