Modified Nelder-Mead Method for High-Dimensional Low-Budget Optimization

Black-box optimization (BBO) is a widely used technique for solving a variety of optimization problems including real-world applications with a high-dimensional expensive objective function. Among BBO methods, the Nelder-Mead (NM) method, which is a local search heuristic using a simplex, has been successful due to its simplicity and practical performance on low-dimensional problems. However, the NM method requires..+ 1 and.. evaluations to perform its initialization and Shrinkage operations respectively to optimize an..-dimensional objective. This is problematic when the objective is computationally and/or financially expensive because, in such a situation, we usually have a limited evaluation budget but those operations consume most of the entire budget. In this study, to address this drawback, we propose a simple but practical modification of the NM method that efficiently works for high-dimensional low-budget optimization. Our numerical results demonstrate that the proposed approach outperforms the original NM method and the random search baselines on BBO benchmark problems.