Automated Performance Benchmarking Platform of IaaS Cloud

With the rapid development of cloud computing, IaaS (Infrastructure as a Service) becomes more and more popular. IaaS customers may not clearly know the actual performance of each cloud platform. Moreover, there are no unified standards in performance evaluation of IaaS VMs (virtual machine). The underlying virtualization technology of IaaS cloud is transparent to customers. In this paper, we will design an automated performance benchmarking platform which can automatically install, configure and execute each benchmarking tool with a configuration center. This platform can easily visualize multidimensional benchmarking parameters data of each IaaS cloud platform. We also rented four IaaS VMs from AliCloud-Beijing, AliCloud-Qingdao, UCloud and Huawei to validate our benchmarking system. Performance comparisons of multiple parameters between multiple platforms were shown in this paper. However, in practice, customers' applications running on VMs are often complex. Performance of complex applications may not depend on single benchmarking parameter (e.g. CPU, memory, disk I/O etc.). We ran a TPC-C test for example to get overall performance in MySQL application scenario. The effects of different benchmarking parameters differ in this specific scenario.