Budget-Aware Scheduling for Hyperparameter Optimization Process in Cloud Environment

Hyperparameter optimization, as a necessary step for majority machine learning models, is crucial to achieving optimal model performance. Unfortunately, the process of hyperparameter optimization is usually computation-intensive and time-consuming due to the large searching space. To date, with the popularity and maturity of cloud computing, many researchers leverage public cloud services (i.e. Amazon AWS) to train machine learning models. Time and monetary cost, two contradictory targets, are what cloud machine learning users are more concerned about. In this paper, we propose HyperWorkflow, a workflow engine service for hyperparameter optimization execution, that coordinates between hyperparameter optimization job and cloud service instances. HyperWorkflow orchestrates the hyperparameter optimization process in a parallel and cost-effective manner upon heterogeneous cloud resources, and schedules hyperparameter trials using bin packing approach to make the best use of cloud resources to speed up the tuning processing under budget constraint. The evaluations show that HyperWorkflow can speed up hyperparameter optimization execution across a range of different budgets.