ON REGULARIZATION AND ACTIVE-SET METHODS WITH COMPLEXITY FOR CONSTRAINED OPTIMIZATION

The main objective of this research is to introduce a practical method for smooth bound-constrained optimization that possesses worst-case evaluation complexity O(epsilon (3/2)) for finding an epsilon-approximate first-order stationary point when the Hessian of the objective function is Lipschitz continuous. As other well-established algorithms for optimization with box constraints, the algorithm proceeds visiting the different faces of the domain aiming to reduce the norm of an internal projected gradient and abandoning active constraints when no additional progress is expected in the current face. The introduced method emerges as a particular case of a method for minimization with linear constraints. Moreover, the linearly constrained minimization algorithm is an instance of a minimization algorithm with general constraints whose implementation may be unaffordable when the constraints are complicated. As a procedure for leaving faces, a different method is employed that may be regarded as an independent device for constrained optimization. Such an independent algorithm may be employed to solve linearly constrained optimization problems on its own, without relying on the active-set strategy. A careful implementation and numerical experiments show that the algorithm that combines active sets with leaving-face iterations is more effective than the independent algorithm on which leaving-face iterations are based, although both exhibit similar complexities O(epsilon (-3/2)).