Infra: SLO Aware Elastic Auto Scaling in the Cloud for Cost Reduction

Enterprises often host applications and services on clusters of virtual machine instances provided by cloud service providers, like Amazon, Rackspace, Microsoft, etc. Users pay a cloud usage cost on the basis of the hourly usage [1] of virtual machine instances composing the cluster. A cluster composition refers to the number of virtual machine instances of each type (from a predefined list of types) comprising a cluster. We present Infra, a cloud provisioning framework that can predict an (epsilon, delta)-minimum cluster composition required to run a given application workload on a cloud under an SLO (i.e., Service Level Objective) deadline. This paper does not present a new approximation algorithm, instead we provide a tool that applies existing machine learning techniques to predict an (epsilon, delta)-minimum cluster composition. An (epsilon, delta)-minimum cluster composition specifies a cluster composition whose cost approximates that of the minimum cluster composition (i.e., the cluster composition that incurs the minimum cloud usage cost that must be incurred in executing a given application under an SLO deadline); the approximation bounds the error to a predefined threshold epsilon with a degree of confidence 100 * (1 - delta)%. The degree of confidence 100 * (1 - delta)% specifies that the probability of failure in achieving the error threshold epsilon for the above approximation is at most delta. For epsilon = 0.1 and delta = 0.02, we experimentally demonstrate that an (epsilon, delta)-minimum cluster composition predicted by Infra successfully approximates the minimum cluster composition, i.e., the accuracy of prediction of minimum cluster composition ranges from 93.1% to 97.99% (the error is bound by the error threshold of 0.1) with a 98% degree of confidence, since 100* (1 - delta) = 98%. Auto scaling refers to the process of automatically adding cloud instances to a cluster to adapt to an increase in application workload (increased request rate), and deleting instances from a cluster when there is a decrease in workload (reduced request rate). However, state-of-the-art auto scaling techniques have the following disadvantages: A) they require explicit policy definition for changing the cluster configuration and therefore lack the ability to automatically adapt a cluster with respect to changing workload, B) they do not compute the appropriate size of resources required, and therefore do not result in an "optimal" cluster composition. Infra provides an auto scaler that automatically adapts a cloud infrastructure to changing application workload, scaling the cluster up/down based on predictions from the Infra provisioning tool.