Gaussian Process Assisted Particle Swarm Optimization

Real-world optimization problems often are non-convex, non-differentiable and highly multimodal, which is why stochastic, population-based metaheuristics are frequently applied. If the optimization problem is also computationally very expensive, only relatively few function evaluations can be afforded. We develop a model-assisted optimization approach as a coupling of Gaussian Process modeling, a regression technique from machine learning, with the Particle Swarm Optimization metaheuristic. It uses earlier function evaluations to predict areas of improvement and exploits the model information in the heuristic search. Under the assumption of a costly target function, it is shown that model-assistance improves the performance across a set of standard benchmark functions. In return, it is possible to reduce the number of target function evaluations to reach a certain fitness level to speed up the search.