Optimizing Distributed Computing Workflows in Heterogeneous Network Environments

Next-generation computation-intensive applications in various science and engineering fields feature large-scale computing workflows. Supporting such computing workflows and optimizing their network performance in terms of end-to-end delay or frame rate in heterogeneous network environments are critical to the success of these distributed applications that require fast response time or smooth data flow. We formulate six linear pipeline configuration problems with different mapping objectives and network constraints, and one general workflow mapping problem. We investigate the computational complexity of these problems and design optimal or heuristic algorithms with rigorous correctness proof and performance analysis. An extensive set of optimization experiments on a large number of simulated workflows and networks illustrate the superior performance of the proposed algorithms in comparison with that of existing methods.