ANALYZING RANDOM PERMUTATIONS FOR CYCLIC COORDINATE DESCENT

被引:9
|
作者
Wright, Stephen J. [1 ]
Lee, Ching-Pei [1 ,2 ]
机构
[1] Univ Wisconsin, Dept Comp Sci, Madison, WI 53706 USA
[2] Natl Univ Singapore, Dept Math, Singapore, Singapore
关键词
Coordinate descent; Gauss-Seidel; randomization; permutations;
D O I
10.1090/mcom/3530
中图分类号
O29 [应用数学];
学科分类号
070104 ;
摘要
We consider coordinate descent methods for minimization of convex quadratic functions, in which exact line searches are performed at each iteration. (This algorithm is identical to Gauss-Seidel on the equivalent symmetric positive definite linear system.) We describe a class of convex quadratic functions for which the random permutations version of cyclic coordinate descent (RPCD) is observed to outperform the standard cyclic coordinate descent (CCD) approach on computational tests, yielding convergence behavior similar to the fully random variant (RCD). A convergence analysis is developed to explain the empirical observations.
引用
收藏
页码:2217 / 2248
页数:32
相关论文
共 50 条
  • [1] Random permutations fix a worst case for cyclic coordinate descent
    Lee, Ching-Pei
    Wright, Stephen J.
    [J]. IMA JOURNAL OF NUMERICAL ANALYSIS, 2019, 39 (03) : 1246 - 1275
  • [2] Randomness and permutations in coordinate descent methods
    Gurbuzbalaban, Mert
    Ozdaglar, Asuman
    Vanli, Nuri Denizcan
    Wright, Stephen J.
    [J]. MATHEMATICAL PROGRAMMING, 2020, 181 (02) : 349 - 376
  • [3] Randomness and permutations in coordinate descent methods
    Mert Gürbüzbalaban
    Asuman Ozdaglar
    Nuri Denizcan Vanli
    Stephen J. Wright
    [J]. Mathematical Programming, 2020, 181 : 349 - 376
  • [4] Descent sets of cyclic permutations
    Elizalde, Sergi
    [J]. ADVANCES IN APPLIED MATHEMATICS, 2011, 47 (04) : 688 - 709
  • [5] When Cyclic Coordinate Descent Outperforms Randomized Coordinate Descent
    Gurbuzbalaban, Mert
    Ozdaglar, Asuman
    Parrilo, Pablo A.
    Vanli, N. Denizcan
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 2017, 30
  • [6] CYCLIC STRUCTURE OF RANDOM PERMUTATIONS
    KOLCHIN, VF
    CHISTYAKOV, VP
    [J]. MATHEMATICAL NOTES, 1975, 18 (5-6) : 1139 - 1144
  • [7] The first descent in samples of geometric random variables and permutations
    Knopfmacher, Arnold
    Prodinger, Helmut
    [J]. DISCRETE MATHEMATICS AND THEORETICAL COMPUTER SCIENCE, 2006, 8 (01): : 215 - 234
  • [8] The last descent in samples of geometric random variables and permutations
    Knopfmacher, Arnold
    Prodinger, Helmut
    [J]. ARS COMBINATORIA, 2007, 83 : 365 - 379
  • [9] Monotonous subsequences and the descent process of invariant random permutations
    Kammoun, Mohamed Slim
    [J]. ELECTRONIC JOURNAL OF PROBABILITY, 2018, 23
  • [10] The first descent in samples of geometric random variables and permutations
    Knopfmacher, Arnold
    Prodinger, Helmut
    [J]. DISCRETE MATHEMATICS AND THEORETICAL COMPUTER SCIENCE, 2008, 10 (01): : 215 - 234