Parallel Primal-Dual Method with Linearization for Structured Convex Optimization

This paper presents the Parallel Primal-Dual (PPD3) algorithm, an innovative approach to solving optimization problems characterized by the minimization of the sum of three convex functions, including a Lipschitz continuous term. The proposed algorithm operates in a parallel framework, simultaneously updating primal and dual variables, and offers potential computational advantages. This parallelization can greatly accelerate computation, particularly when run on parallel computing platforms. By departing from traditional primal-dual methods that necessitate strict parameter constraints, the PPD3 algorithm removes reliance on the spectral norm of the linear operator, significantly reducing the computational burden associated with its evaluation. As the problem size grows, calculating the spectral norm, which is essential for many primal-dual methods, becomes progressively more expensive. In addition, adaptive step sizes are computed to accelerate the convergence process. In contrast to most primal-dual approaches that employ a fixed step size constrained by a global upper limit throughout all iterations, the adaptive step size is typically greater and may result in faster convergence. An O(1/k) ergodic convergence rate is proved theoretically. Applications in Fused LASSO and image inpainting demonstrate the method's efficiency in computation time and convergence rate compared to state-of-the-art algorithms.