An Implementation of a Multi-site Virtual Cluster Cloud

The use of virtual cluster for High Performance Computing (HPC) work has a great benefit of hiding the complexity of physical infrastructure while providing a familiar HPC environment for users. Nevertheless, setting up one is still a challenging task for most users. In addition, the ability to span a virtual cluster across a multi-site cloud can even create a great potential for users to harness a much higher processing power from the cloud. In this paper, the development of a multi-site virtual cluster cloud is presented. This is done by providing the cloud infrastructures tailored specifically for virtual cluster deployment along with related utilities such as web-interface monitor and command line tools. To realize this infrastructure, an overlay network is created using Virtual Private Network (VPN) technology to merge every site into a single broadcast domain. Moreover, a Multi-tier VPN network topology is proposed to enhance the communication performance among the multi-site cloud. A smart virtual machine scheduling mechanism is introduced to reduce performance degradation caused by network delays. The experimental results show that the virtual cluster works well with highly parallel task with low communication among task. Thus, this technology can be used to support many classes of important HPC applications.