Efficient Data and Task Co-Scheduling for Scientific Workflow in Geo-distributed Datacenters

被引：5
|
作者：
Chen, Jian [1 ]
Zhang, Jinghui [1 ]
Song, Aibo [1 ]
机构：
[1] Southeast Univ, Sch Comp Sci & Engn, Nanjing 211189, Jiangsu, Peoples R China
来源：
2017 FIFTH INTERNATIONAL CONFERENCE ON ADVANCED CLOUD AND BIG DATA (CBD) | 2017年
基金：
中国国家自然科学基金;
关键词：
datacenter; scientific workflow scheduling; graph partition; linear programming; DEDICATED HETEROGENEOUS MULTICLUSTER; STRATEGY;
D O I：
10.1109/CBD.2017.19
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
Scientific workflow usually needs to be performed in multiple collaborative datacenters for the requirement of accessing community-wide resources. However, the movements of initial input data and intermediate data across geo-distributed datacenters would hinder efficient execution of large-scale dataintensive scientific workflows. In this paper, a novel scheduling approach based on graph partition is proposed for the execution of data-intensive scientific workflow in geo-distributed datacenters, aiming at the optimization of the overall data transfer cost. Simulations show that our algorithm significantly reduces the overall geo-distributed data transfer and demonstrate its effectiveness.
引用
收藏
页码：63 / 68
页数：6
相关论文
共 50 条
  • [1] Graph partition-based data and task co-scheduling of scientific workflow in geo-distributed datacenters
    Zhang, Jinghui
    Chen, Jian
    Zhan, Jun
    Jin, Jiahui
    Song, Aibo
    [J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2019, 31 (24):
  • [2] Scheduling Jobs Across Geo-distributed Datacenters
    Hung, Chien-Chun
    Golubchik, Leana
    Yu, Minlan
    [J]. ACM SOCC'15: PROCEEDINGS OF THE SIXTH ACM SYMPOSIUM ON CLOUD COMPUTING, 2015, : 111 - 124
  • [3] Flutter: Scheduling Tasks Closer to Data Across Geo-Distributed Datacenters
    Hu, Zhiming
    Li, Baochun
    Luo, Jun
    [J]. IEEE INFOCOM 2016 - THE 35TH ANNUAL IEEE INTERNATIONAL CONFERENCE ON COMPUTER COMMUNICATIONS, 2016,
  • [4] On Achieving Efficient Data Transfer for Graph Processing in Geo-Distributed Datacenters
    Zhou, Amelie Chi
    Ibrahim, Shadi
    He, Bingsheng
    [J]. 2017 IEEE 37TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS (ICDCS 2017), 2017, : 1397 - 1407
  • [5] Fault-tolerant scheduling and data placement for scientific workflow processing in geo-distributed clouds
    Li, Chunlin
    Liu, Jun
    Wang, Min
    Luo, Youlong
    [J]. JOURNAL OF SYSTEMS AND SOFTWARE, 2022, 187
  • [6] MapReduce Task Scheduling in Heterogeneous Geo-Distributed Data Centers
    Li, Xiaoping
    Chen, Fuchao
    Ruiz, Ruben
    Zhu, Jie
    [J]. IEEE TRANSACTIONS ON SERVICES COMPUTING, 2022, 15 (06) : 3317 - 3329
  • [7] Privacy-preserving workflow scheduling in geo-distributed data centers
    Xiao, Yao
    Zhou, Amelie Chi
    Yang, Xuan
    He, Bingsheng
    [J]. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2022, 130 : 46 - 58
  • [8] On efficient virtual cluster scaling across geo-distributed datacenters
    Xu, Xinping
    Li, Wenxin
    Qi, Heng
    Li, Keqiu
    [J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2018, 30 (10):
  • [9] Time- and Cost- Efficient Task Scheduling across Geo-Distributed Data Centers
    Hu, Zhiming
    Li, Baochun
    Luo, Jun
    [J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2018, 29 (03) : 705 - 718
  • [10] Efficient Graph Query Processing over Geo-Distributed Datacenters
    Yuan, Ye
    Ma, Delong
    Wen, Zhenyu
    Ma, Yuliang
    Wang, Guoren
    Chen, Lei
    [J]. PROCEEDINGS OF THE 43RD INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL (SIGIR '20), 2020, : 619 - 628
12345