New scaling on the gradient method

Movtivated by the idea of Barzilai and Borwein (BB) method, we introduce a concept of approximately optimal stepsize and present a new gradient method with approximately optimal stepsize for unconstrained optimization. We construct a new quadratic approximation model to generate an approximately optimal stepsize. We then use the two BB stepsizes to truncate it and treat the resulted approximately optimal stepsize as the new stepsize for gradient method. Moreover, for the nonconvex case, we also design two approximate models to generate approximately optimal stepsizes for gradient method. We analyze the convergence of the proposed method. The numerical results show that the proposed method is very promising.