An adaptive ensemble of surrogate models based on heuristic model screening

Ensembles of surrogate models have received increasing attention due to their more robust performance than that of individual surrogate models (ISMs) in the face of different problems. In this work, a novel adaptive ensemble of surrogate models based on heuristic model screening (AE-HMS) is proposed. First, a performance index (PI) combining a distance measure (DM) and cross validation (CV) is employed to determine the performance of the ISMs. Second, a heuristic model screening method based on the PI is used to select acceptable ISMs and eliminate poor ISMs. Compared with previous model screening methods, the proposed heuristic model screening method can better eliminate ISMs with poor performance. Finally, the weight factor of the baseline model (the ISM with the smallest PI) is adaptively allocated according to its PI, and the weight factors of the other ISMs are calculated in a point-by-point manner to complete the ensemble construction process. Based on this process and three representative DMs, three variations of the AE-HMS are proposed. A total of 42 test functions are used to select the appropriate AE-HMS hyperparameters and evaluate its accuracy and robustness. The results show that the AE-HMS has higher accuracy and stronger robustness than the ISMs and other ensembles. More importantly, the same results are obtained in an optimization problem concerning a safety valve, indicating that this model can provide an effective design optimization method for engineering problems.