Randomized Primal-Dual Proximal Block Coordinate Updates

被引:14
|
作者
Gao, Xiang [1 ]
Xu, Yang-Yang [2 ]
Zhang, Shu-Zhong [1 ]
机构
[1] Univ Minnesota, Dept Ind & Syst Engn, Minneapolis, MN USA
[2] Rensselaer Polytech Inst, Dept Math Sci, Troy, NY 12180 USA
基金
美国国家科学基金会;
关键词
Primal-dual method; Alternating direction method of multipliers (ADMM); Randomized algorithm; Iteration complexity; First-order stochastic approximation;
D O I
10.1007/s40305-018-0232-4
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
In this paper, we propose a randomized primal-dual proximal block coordinate updating framework for a general multi-block convex optimization model with coupled objective function and linear constraints. Assuming mere convexity, we establish its O(1/t) convergence rate in terms of the objective value and feasibility measure. The framework includes several existing algorithms as special cases such as a primal-dual method for bilinear saddle-point problems (PD-S), the proximal Jacobian alternating direction method of multipliers (Prox-JADMM) and a randomized variant of the ADMM for multi-block convex optimization. Our analysis recovers and/or strengthens the convergence properties of several existing algorithms. For example, for PD-S our result leads to the same order of convergence rate without the previously assumed boundedness condition on the constraint sets, and for Prox-JADMM the new result provides convergence rate in terms of the objective value and the feasibility violation. It is well known that the original ADMM may fail to converge when the number of blocks exceeds two. Our result shows that if an appropriate randomization procedure is invoked to select the updating blocks, then a sublinear rate of convergence in expectation can be guaranteed for multi-block ADMM, without assuming any strong convexity. The new approach is also extended to solve problems where only a stochastic approximation of the subgradient of the objective is available, and we establish an O(1/convergence rate of the extended approach for solving stochastic programming.
引用
收藏
页码:205 / 250
页数:46
相关论文
共 50 条
  • [1] Randomized Primal–Dual Proximal Block Coordinate Updates
    Xiang Gao
    Yang-Yang Xu
    Shu-Zhong Zhang
    Journal of the Operations Research Society of China, 2019, 7 : 205 - 250
  • [2] A New Randomized Block-Coordinate Primal-Dual Proximal Algorithm for Distributed Optimization
    Latafat, Puya
    Freris, Nikolaos M.
    Patrinos, Panagiotis
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2019, 64 (10) : 4050 - 4065
  • [3] DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization
    Xiao, Lin
    Yu, Adams Wei
    Lin, Qihang
    Chen, Weizhu
    JOURNAL OF MACHINE LEARNING RESEARCH, 2019, 20
  • [4] Accelerated primal-dual proximal block coordinate updating methods for constrained convex optimization
    Xu, Yangyang
    Zhang, Shuzhong
    COMPUTATIONAL OPTIMIZATION AND APPLICATIONS, 2018, 70 (01) : 91 - 128
  • [5] DSCOVR: Randomized primal-dual block coordinate algorithms for asynchronous distributed optimization
    Xiao, Lin
    Yu, Adams Wei
    Lin, Qihang
    Chen, Weizhu
    Journal of Machine Learning Research, 2019, 20
  • [6] A RANDOM BLOCK-COORDINATE PRIMAL-DUAL PROXIMAL ALGORITHM WITH APPLICATION TO 3D MESH DENOISING
    Repetti, Audrey
    Chouzenoux, Emilie
    Pesquet, Jean-Christophe
    2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 2015, : 3561 - 3565
  • [7] A PARALLEL BLOCK-COORDINATE APPROACH FOR PRIMAL-DUAL SPLITTING WITH ARBITRARY RANDOM BLOCK SELECTION
    Repetti, Audrey
    Chouzenoux, Emilie
    Pesquet, Jean-Christophe
    2015 23RD EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO), 2015, : 235 - 239
  • [8] Random extrapolation for primal-dual coordinate descent
    Alacaoglu, Ahmet
    Fercoq, Olivier
    Cevher, Volkan
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 119, 2020, 119
  • [9] INERTIAL, CORRECTED, PRIMAL-DUAL PROXIMAL SPLITTING
    Valkonen, Tuomo
    SIAM JOURNAL ON OPTIMIZATION, 2020, 30 (02) : 1391 - 1420
  • [10] Block-Coordinate Primal-Dual Method for Nonsmooth Minimization over Linear Constraints
    Luke, D. Russell
    Malitsky, Yura
    LARGE-SCALE AND DISTRIBUTED OPTIMIZATION, 2018, 2227 : 121 - 147