Taking another step: A simple approach to high-dimensional Bayesian optimization

Scaling Bayesian optimization to high-dimensional problems is a meaningful but challenging task. Most of current approaches assume only a few variables are effective to the objective function or the objective function is additively separable, therefore are not suitable when the problem violates these assumptions. In this work, we propose a simple and efficient approach to extend Bayesian optimization to high dimensions. The proposed approach does not make these two assumptions. In each iteration, after locating a candidate point by the global Gaussian process model, we train a local Gaussian process model around the candidate point and locate a new point using the local model. Instead of evaluating the solution located by the global model, we evaluate the solution located by the local model. This simple taking-another-step approach is shown to be able to improve Bayesian optimization's performance significantly on high-dimensional optimization problems. The proposed algorithm also shows competitive performance when compared with four high-dimensional Bayesian optimization algorithms and four surrogate-assisted evolutionary algorithms.