Accelerated methods with fastly vanishing subgradients for structured non-smooth minimization

被引:1
|
作者
Mainge, Paul-Emile [1 ]
Labarre, Florian [1 ]
机构
[1] Univ Antilles, MEMIAD, Campus Schoelcher, F-97233 Schoelcher, Martinique, France
关键词
Nesterov-type algorithm; Inertial-type algorithm; Global rate of convergence; Non-smooth minimization; Structured minimization; Fast first-order method; CONVEX-OPTIMIZATION; DYNAMICAL-SYSTEM; CONVERGENCE; ALGORITHMS; SUM;
D O I
10.1007/s11075-021-01181-y
中图分类号
O29 [应用数学];
学科分类号
070104 ;
摘要
In a real Hilbert space, we study a new class of forward-backward algorithms for structured non-smooth minimization problems. As a special case of the parameters, we recover the method AFB (Accelerated Forward-Backward) that was recently discussed as an enhanced variant of FISTA (Fast Iterative Soft Thresholding Algorithm). Our algorithms enjoy the well-known properties of AFB. Namely, they generate convergent sequences (x(n)) that minimize the function values at the rate o(n(- 2)). Another important specificity of our processes is that they can be regarded as discrete models suggested by first-order formulations of Newton-like dynamical systems. This permit us to extend to the non-smooth setting, a property of fast convergence to zero of the gradients, established so far for discrete Newton-like dynamics with smooth potentials only. In specific, as a new result, we show that the latter property also applies to AFB. To prove this stability phenomenon, we develop a technical analysis that can be also useful regarding many other related developments. Numerical experiments are furthermore performed so as to illustrate the properties of the considered algorithms comparing with other existing ones.
引用
收藏
页码:99 / 136
页数:38
相关论文
共 50 条
  • [1] Accelerated methods with fastly vanishing subgradients for structured non-smooth minimization
    Paul-Emile Maingé
    Florian Labarre
    [J]. Numerical Algorithms, 2022, 90 : 99 - 136
  • [2] Smooth minimization of non-smooth functions
    Nesterov, Y
    [J]. MATHEMATICAL PROGRAMMING, 2005, 103 (01) : 127 - 152
  • [3] Smooth minimization of non-smooth functions
    Yu. Nesterov
    [J]. Mathematical Programming, 2005, 103 : 127 - 152
  • [4] A MODIFICATION AND AN EXTENSION OF LEMARECHAL ALGORITHM FOR NON-SMOOTH MINIMIZATION
    MIFFLIN, R
    [J]. MATHEMATICAL PROGRAMMING STUDY, 1982, 17 (APR): : 77 - 90
  • [5] Accelerated convergence for Schrodinger equations with non-smooth potentials
    Kieri, Emil
    [J]. BIT NUMERICAL MATHEMATICS, 2014, 54 (03) : 729 - 748
  • [6] Minimization of Non-smooth, Non-convex Functionals by Iterative Thresholding
    Bredies, Kristian
    Lorenz, Dirk A.
    Reiterer, Stefan
    [J]. JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 2015, 165 (01) : 78 - 112
  • [7] Smooth over-parameterized solvers for non-smooth structured optimization
    Clarice Poon
    Gabriel Peyré
    [J]. Mathematical Programming, 2023, 201 : 897 - 952
  • [8] A splitting bundle approach for non-smooth non-convex minimization
    Fuduli, A.
    Gaudioso, M.
    Nurminski, E. A.
    [J]. OPTIMIZATION, 2015, 64 (05) : 1131 - 1151
  • [9] Smooth over-parameterized solvers for non-smooth structured optimization
    Poon, Clarice
    Peyre, Gabriel
    [J]. MATHEMATICAL PROGRAMMING, 2023, 201 (1-2) : 897 - 952
  • [10] Minimization of Non-smooth, Non-convex Functionals by Iterative Thresholding
    Kristian Bredies
    Dirk A. Lorenz
    Stefan Reiterer
    [J]. Journal of Optimization Theory and Applications, 2015, 165 : 78 - 112