Complexity of a projected Newton-CG method for optimization with bounds

被引:1
|
作者
Xie, Yue [1 ,2 ]
Wright, Stephen J. [3 ]
机构
[1] Univ Hong Kong, Dept Math, Pokfulam, Hong Kong, Peoples R China
[2] Univ Hong Kong, Musketeers Fdn Inst Data Sci, Pokfulam, Hong Kong, Peoples R China
[3] Univ Wisconsin, Comp Sci Dept, 1210 W Dayton St, Madison, WI 53706 USA
关键词
Nonconvex bound-constrained optimization; Complexity guarantees; Projected gradient method; Newton's method; Conjugate gradient method; NONNEGATIVE MATRIX FACTORIZATION; NONCONVEX OPTIMIZATION; REGULARIZATION; ALGORITHM;
D O I
10.1007/s10107-023-02000-z
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
This paper describes a method for solving smooth nonconvex minimization problems subject to bound constraints with good worst-case complexity guarantees and practical performance. The method contains elements of two existing methods: the classi-cal gradient projection approach for bound-constrained optimization and a recently proposed Newton-conjugate gradient algorithm for unconstrained nonconvex opti-mization. Using a new definition of approximate second-order optimality parametrized by some tolerance E (which is compared with related definitions from previous works), we derive complexity bounds in terms of E for both the number of iterations required and the total amount of computation. The latter is measured by the number of gradient evaluations or Hessian-vector products. We also describe illustrative computational results on several test problems from low-rank matrix optimization.
引用
收藏
页码:107 / 144
页数:38
相关论文
共 50 条
  • [1] A projected Newton-CG method for nonnegative astronomical image deblurring
    G. Landi
    E. Loli Piccolomini
    [J]. Numerical Algorithms, 2008, 48 : 279 - 300
  • [2] A projected Newton-CG method for nonnegative astronomical image deblurring
    Landi, G.
    Piccolomini, E. Loli
    [J]. NUMERICAL ALGORITHMS, 2008, 48 (04) : 279 - 300
  • [3] A Newton-CG algorithm with complexity guarantees for smooth unconstrained optimization
    Royer, Clement W.
    O'Neill, Michael
    Wright, Stephen J.
    [J]. MATHEMATICAL PROGRAMMING, 2020, 180 (1-2) : 451 - 488
  • [4] A Newton-CG algorithm with complexity guarantees for smooth unconstrained optimization
    Clément W. Royer
    Michael O’Neill
    Stephen J. Wright
    [J]. Mathematical Programming, 2020, 180 : 451 - 488
  • [5] A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees
    O'Neill, Michael
    Wright, Stephen J.
    [J]. IMA JOURNAL OF NUMERICAL ANALYSIS, 2021, 41 (01) : 84 - 121
  • [6] Inexact Newton-CG algorithms with complexity guarantees
    Yao, Zhewei
    Xu, Peng
    Roosta, Fred
    Wright, Stephen J.
    Mahoney, Michael W.
    [J]. IMA JOURNAL OF NUMERICAL ANALYSIS, 2023, 43 (03) : 1855 - 1897
  • [7] An active set Newton-CG method for l1 optimization
    Cheng, Wanyou
    Dai, Yu-Hong
    [J]. APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS, 2021, 50 : 303 - 325
  • [8] Linesearch Newton-CG methods for convex optimization with noise
    Bellavia S.
    Fabrizi E.
    Morini B.
    [J]. ANNALI DELL'UNIVERSITA' DI FERRARA, 2022, 68 (2) : 483 - 504
  • [9] A NEWTON-CG AUGMENTED LAGRANGIAN METHOD FOR SEMIDEFINITE PROGRAMMING
    Zhao, Xin-Yuan
    Sun, Defeng
    Toh, Kim-Chuan
    [J]. SIAM JOURNAL ON OPTIMIZATION, 2010, 20 (04) : 1737 - 1765
  • [10] A NEWTON-CG BASED BARRIER METHOD FOR FINDING A SECOND-ORDER STATIONARY POINT OF NONCONVEX CONIC OPTIMIZATION WITH COMPLEXITY GUARANTEES
    He, Chuan
    Lu, Zhaosong
    [J]. SIAM JOURNAL ON OPTIMIZATION, 2023, 33 (02) : 1191 - 1222