CORP: Cooperative Opportunistic Resource Provisioning for Short-Lived Jobs in Cloud Systems

In cloud systems, achieving high resource utilization and low Service Level Objective (SLO) violation rate are important to the cloud provider for high profit. For this purpose, recently, some methods have been proposed to predict allocated but unused resources and reallocate them to long-running service jobs. However, the accuracy of their prediction method relies on the existence of patterns in jobs' resource utilization. Therefore, these methods cannot be used for short-lived jobs, which usually do not have certain patterns but exhibit frequent fluctuations in resource requirements. Also, these methods may result in resource fragmentation and lead to low resource utilization because they neglect job resource intensity in multi-resource allocation and may allocate much more resources to jobs. To handle this problem, we propose a Cooperative Opportunistic Resource Provisioning scheme (CORP) for short-lived jobs. CORP uses the deep learning method to predict the amount of temporarily-unused resource of each short-lived job. It also predicts the fluctuations of the amount of unused resource using Hidden Markov Model, and adjusts the predicted amount for the peak and valley of unused resource, and dynamically allocates the corrected amount of resource to jobs. Further, CORP uses a job packing strategy by leveraging complementary jobs' requirements on different resource types and allocates such jobs to the same VM to fully utilize unused resources, which increases resource utilization. Extensive experimental results based on a real cluster and Amazon EC2 show that CORP achieves high resource utilization and low SLO violation rate compared to previous resource provisioning schemes.