A novel hybrid trust region algorithm based on nonmonotone and LOOCV techniques

In this paper, a novel hybrid trust-region algorithm using radial basis function (RBF) interpolations is proposed. The new algorithm is an improved version of ORBIT algorithm based on two novel ideas. Because the accuracy and stability of RBF interpolation depends on a shape parameter, so it is more appropriate to select this parameter according to the optimization problem. In the new algorithm, the appropriate shape parameter value is determined according to the optimization problem based on an effective statistical approach, while the ORBIT algorithm in all problems uses a fixed shape parameter value. In addition, the new algorithm is equipped with a new intelligent nonmonotone strategy which improves the speed of convergence, while the monotonicity of the sequence of objective function values in the ORBIT may decrease the rate of convergence, especially when an iteration is trapped near a narrow curved valley. The global convergence of the new hybrid algorithm is analyzed under some mild assumptions. The numerical results significantly indicate the superiority of the new algorithm compared with the original version.