A golden ratio proximal alternating direction method of multipliers for separable convex optimization

Separable convex optimization problems often arise from large scale applications, and alternating direction method of multipliers (ADMM), due to its ability to utilize the separable structure of the objective function, has become an extremely popular approach for solving this class of problems. However, the convergence of the primal iterates generated by ADMM is not guaranteed and the ADMM subproblems can be computationally demanding. Proximal ADMM (PADMM), which introduces proximal terms to the ADMM subproblems, not only guarantees convergence of both the primal and the dual iterates but also is able to take advantage of the problem structures. In this paper, by adopting a convex combination technique we propose a new variant of the classical ADMM, which we call golden ratio proximal ADMM (GrpADMM) as the golden ratio appears to be a key parameter. GrpADMM preserves all the favorable features of PADMM, such as the ability to take full use of problem structures and global convergence under relaxed parameter condition. We show that GrpADMM shares the O(1/N)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal {O}}({1}/{N})$$\end{document} ergodic sublinear convergence rate, where N denotes the iteration counter. Furthermore, as long as one of the functions in the objective is strongly convex, the algorithm can be modified to achieve faster O(1/N2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal {O}}(1/N^2)$$\end{document} ergodic convergence. Finally, we demonstrate the performance of the proposed algorithms via preliminary numerical experiments.