An Integrated High-Performance Transport Solution for Big Data Transfer over Wide-area Networks

Transport control plays an important role in the performance of large-scale scientific applications involving transfer of large data sets, online computational steering, and interactive visualization. In general, large-scale scientific applications require high bandwidths to move big data across wide-area networks. One of the primary challenges is to explore and compose a set of feasible route options with multiple constraints. Another challenge essentially arises from the randomness inherent in wide-area networks, particularly the Internet. This randomness must be explicitly accounted for to achieve both goodput maximization and stabilization over the constructed routes by suitably adjusting the source rate in response to both network and host dynamics. Unfortunately, the widely deployed Transmission Control Protocol is inadequate for such tasks due to its performance limitations. We conduct rigorous analytical study of the design and performance of transport solutions, and develop an integrated transport solution in a systematical way to overcome the limitations of current transport methods. The superior and robust performance of the proposed transport solution is extensively evaluated in an experimental environment in comparison with existing transport methods.