Solving saddle point problems: a landscape of primal-dual algorithm with larger stepsizes

We consider a class of saddle point problems frequently arising in the areas of image processing and machine learning. In this paper, we propose a simple primal-dual algorithm, which embeds a general proximal term induced with a positive definite matrix into one subproblem. It is remarkable that our algorithm enjoys larger stepsizes than many existing state-of-the-art primal-dual-like algorithms due to our relaxed convergence-guaranteeing condition. Moreover, our algorithm includes the well-known primal-dual hybrid gradient method as its special case, while it is also of possible benefit to deriving partially linearized primal-dual algorithms. Finally, we show that our algorithm is able to deal with multi-block separable saddle point problems. In particular, an application to a multi-block separable minimization problem with linear constraints yields a parallel algorithm. Some computational results sufficiently support the promising improvement brought by our relaxed requirement.