A Study on Speculative Task Scheduling for Apache Spark in Fog Computing Realms

Speculative task scheduling is a technique used by most of today's distributed data processing systems to cope with slow running tasks that delay subsequent computations and hamper the overall performance of analytics jobs. Fog computing is now becoming a prominent service paradigm to foster applications with low-latency analytics requirements by moving the analysis closer to the data sources. However, task scheduling in fog realms is not without challenges, as resource heterogeneity and highly volatile network connections have the potential to delay instead of improving the completion time of analytics jobs. In this paper, we present a comprehensive study showcasing the various obstacles that are presented in speculative task scheduling for analytics jobs deployed in fog realms and derive critical insights that must be taken into account when introducing new task scheduling algorithms.