An Efficient Gradient Method with Approximately Optimal Stepsize Based on Tensor Model for Unconstrained Optimization

A new type of stepsize, which was recently introduced by Liu et al. (Optimization 67(3):427–440, 2018), is called approximately optimal stepsize and is very efficient for gradient method. Interestingly, all gradient methods can be regarded as gradient methods with approximately optimal stepsizes. In this paper, we present an efficient gradient method with approximately optimal stepsize based on tensor model for unconstrained optimization. In the proposed method, if the objective function is not close to a minimizer and a quadratic function on a line segment between the current and latest iterates, then a tensor model is exploited to generate approximately optimal stepsize for gradient method. Otherwise, quadratic approximation models are constructed to generate approximately optimal stepsizes for gradient method. The global convergence of the proposed method is established under weak conditions. Numerical results indicate that the proposed method is very promising.