Improving Performance of Distributed Graph Traversals via Application-Aware Plug-In Work Scheduler

Unordered graph algorithms can offer efficient resource utilization that is advantageous for performance in distributed setting. Unordered execution allows for parallel computation without synchronization. In unordered algorithms, work is data-driven and can be performed in any order, refining the result as the algorithm progresses. Unfortunately, a sub-optimal work ordering may lead to more time spent on correcting the results than on useful work. On HPC systems, the issue is compounded by irregular nature of distributed graph algorithms which makes them sensitive to the whole software/hardware stack, collectively referred to as runtime. In this paper, we consider an example of such algorithms: Distributed Control (DC) single-source shortest paths (SSSP). DC relies on performance gains stemming from the inherent asynchrony of unordered algorithms while optimizing work ordering locally. We demonstrate that distributed runtime scheduling policy can prevent effective work ordering optimization. We show that lifting and delegating some scheduling decisions to the algorithm level can result in significantly better performance. We propose that this strategy can be useful for performance engineering.