Accelerated methods with fastly vanishing subgradients for structured non-smooth minimization

被引:1
|
作者
Mainge, Paul-Emile [1 ]
Labarre, Florian [1 ]
机构
[1] Univ Antilles, MEMIAD, Campus Schoelcher, F-97233 Schoelcher, Martinique, France
关键词
Nesterov-type algorithm; Inertial-type algorithm; Global rate of convergence; Non-smooth minimization; Structured minimization; Fast first-order method; CONVEX-OPTIMIZATION; DYNAMICAL-SYSTEM; CONVERGENCE; ALGORITHMS; SUM;
D O I
10.1007/s11075-021-01181-y
中图分类号
O29 [应用数学];
学科分类号
070104 ;
摘要
In a real Hilbert space, we study a new class of forward-backward algorithms for structured non-smooth minimization problems. As a special case of the parameters, we recover the method AFB (Accelerated Forward-Backward) that was recently discussed as an enhanced variant of FISTA (Fast Iterative Soft Thresholding Algorithm). Our algorithms enjoy the well-known properties of AFB. Namely, they generate convergent sequences (x(n)) that minimize the function values at the rate o(n(- 2)). Another important specificity of our processes is that they can be regarded as discrete models suggested by first-order formulations of Newton-like dynamical systems. This permit us to extend to the non-smooth setting, a property of fast convergence to zero of the gradients, established so far for discrete Newton-like dynamics with smooth potentials only. In specific, as a new result, we show that the latter property also applies to AFB. To prove this stability phenomenon, we develop a technical analysis that can be also useful regarding many other related developments. Numerical experiments are furthermore performed so as to illustrate the properties of the considered algorithms comparing with other existing ones.
引用
收藏
页码:99 / 136
页数:38
相关论文
共 50 条
  • [31] THE STUDY OF NON-SMOOTH MAPS IN HILBERT-SPACES BY METHODS OF SMOOTH ANALYSIS
    ANTONOVSKII, EY
    [J]. RUSSIAN MATHEMATICAL SURVEYS, 1980, 35 (03) : 161 - 165
  • [32] Descent Methods for Mixed Variational Inequalities with Non-Smooth Mappings
    Konnov, Igor V.
    [J]. OPTIMIZATION THEORY AND RELATED TOPICS, 2012, 568 : 121 - 138
  • [33] Numerical methods for a Volterra integral equation with non-smooth solutions
    Diogo, T
    Ford, NJ
    Lima, P
    Valtchev, S
    [J]. JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2006, 189 (1-2) : 412 - 423
  • [34] Wasserstein Gradient Flow of the Fisher Information from a Non-Smooth Convex Minimization Viewpoint
    Carlier, Guillaume
    Benamou, Jean-David
    Matthes, Daniel
    [J]. JOURNAL OF CONVEX ANALYSIS, 2024, 31 (02) : 359 - 378
  • [35] Inertial Block Proximal Methods For Non-Convex Non-Smooth Optimization
    Le Thi Khanh Hien
    Gillis, Nicolas
    Patrinos, Panagiotis
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 119, 2020, 119
  • [36] Effective Proximal Methods for Non-convex Non-smooth Regularized Learning
    Liang, Guannan
    Tong, Qianqian
    Ding, Jiahao
    Pan, Miao
    Bi, Jinbo
    [J]. 20TH IEEE INTERNATIONAL CONFERENCE ON DATA MINING (ICDM 2020), 2020, : 342 - 351
  • [37] Inexact Proximal Gradient Methods for Non-Convex and Non-Smooth Optimization
    Gu, Bin
    Wang, De
    Huo, Zhouyuan
    Huang, Heng
    [J]. THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTIETH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / EIGHTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2018, : 3093 - 3100
  • [38] On the rate of convergence of alternating minimization for non-smooth non-strongly convex optimization in Banach spaces
    Both, Jakub Wiktor
    [J]. OPTIMIZATION LETTERS, 2022, 16 (02) : 729 - 743
  • [39] AN EXACT PENALTY-FUNCTION ALGORITHM FOR NON-SMOOTH CONVEX CONSTRAINED MINIMIZATION PROBLEMS
    KIWIEL, KC
    [J]. IMA JOURNAL OF NUMERICAL ANALYSIS, 1985, 5 (01) : 111 - 119
  • [40] Generalized non-convex non-smooth sparse and low rank minimization using proximal average
    Lin, Xiaofan
    Wei, Gang
    [J]. NEUROCOMPUTING, 2016, 174 : 1116 - 1124