Efficient Real-time Earliest Deadline First based scheduling for Apache Spark

被引：3

作者：

Neciu, Laurentiu-Florin ^{[1
]}

Pop, Florin ^{[1
,2
]}

Apostol, Elena-Simona ^{[1
]}

Truica, Ciprian-Octavian ^{[1
]}

机构：

[1] Univ Politehn Bucuresti, Bucharest, Romania

[2] Natl Inst Res & Dev Informat ICI Bucharest, Bucharest, Romania

来源：

2021 20TH INTERNATIONAL SYMPOSIUM ON PARALLEL AND DISTRIBUTED COMPUTING (ISPDC) | 2021年

关键词：

Real-time Scheduling; Apache Spark; Real-Time Earliest Deadline First; Resilient Distributed Dataset; Directed Acyclic Graph;

D O I：

10.1109/ISPDC52870.2021.9521640

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

Apache Spark is a distributed computing framework for fast in-memory data analysis, Machine Learning jobs, and SQL queries that employs Resilient Distributed Dataset for distributing data and Directed Acyclic Graph for scheduling computations. Currently, Spark provides two scheduling policies for tasks pending execution: a First In First Out policy and a FAIR policy, providing no support for deadline-based real-time scheduling. In this paper, we present a new system designed to accept deadlines for heterogeneous Spark Jobs and perform a real-time scheduling policy based on Earliest Deadline First (EDF). To showcase the efficiency of our scheduling policy, we compare and analyze the performance of our solution with the current Spark execution policies in terms of job lateness. We empirically prove that our real-time policy provides much lower lateness given suitable

引用

页码：97 / 104

页数：8

共 50 条

[21] The Global Limited Preemptive Earliest Deadline First Feasibility of Sporadic Real-time Tasks
Thekkilakattil, Abhilash
Baruah, Sanjoy
Dobrin, Radu
Punnekkat, Sasikumar
[J]. 2014 26TH EUROMICRO CONFERENCE ON REAL-TIME SYSTEMS (ECRTS 2014), 2014, : 301 - +
[22] Fast real-time job selection with resource constraints under earliest deadline first
Han, S
Park, M
Cho, Y
[J]. EMBEDDED COMPUTER SYSTEMS: ARCHITECTURES, MODELING, AND SIMULATION, 2005, 3553 : 242 - 250
[23] Real-time queues in heavy traffic with earliest-deadline-first queue discipline
Doytchinov, B
Lehoczky, J
Shreve, S
[J]. ANNALS OF APPLIED PROBABILITY, 2001, 11 (02): : 332 - 378
[24] Quantum-based earliest deadline first scheduling for multiservices
Lim, Teck Meng
Lee, Bu-Sung
Yeo, Chai Kiat
[J]. IEEE TRANSACTIONS ON MULTIMEDIA, 2007, 9 (01) : 157 - 168
[25] Efficient Scheduling of Periodic, Aperiodic, and Sporadic Real-Time Tasks with Deadline Constraints
Goubaa, Aicha
Kahlgui, Mohamed
Georg, Frey
Li, Zhiwu
[J]. SOFTWARE TECHNOLOGIES (ICSOFT 2020), 2021, 1447 : 25 - 43
[26] Deadline-based scheduling in support of real-time data delivery
Liu, Yanni Ellen
Wong, Johnny W.
[J]. COMPUTER NETWORKS, 2008, 52 (03) : 514 - 530
[27] Enabling deadline scheduling for Java real-time computing
CS Technologies Informatiques, Bron, France
[J]. Proc Real Time Syst Symp, 1600, (152-153):
[28] Is Deadline Oblivious Scheduling Efficient for Controlling Real-Time Traffic in Cellular Downlink Systems?
ElAzzouni, Sherif
Ekici, Eylem
Shroff, Ness
[J]. IEEE INFOCOM 2020 - IEEE CONFERENCE ON COMPUTER COMMUNICATIONS, 2020, : 49 - 58
[29] Task Scheduling Algorithm for Heterogeneous Real-time Systems Based on Deadline Constraints
Li, Jianpeng
Zheng, Gewei
Zhang, Hanbing
Shi, Guozhen
[J]. PROCEEDINGS OF 2019 IEEE 9TH INTERNATIONAL CONFERENCE ON ELECTRONICS INFORMATION AND EMERGENCY COMMUNICATION (ICEIEC 2019), 2019, : 113 - 116
[30] On earliest deadline first scheduling for temporal consistency maintenance
Xiong, Ming
Wang, Qiong
Ramamritham, Krithi
[J]. REAL-TIME SYSTEMS, 2008, 40 (02) : 208 - 237

← 1 2 3 4 5 →