IMRO: A PROXIMAL QUASI-NEWTON METHOD FOR SOLVING l1-REGULARIZED LEAST SQUARES PROBLEMS

被引：15

作者：

Karimi, Sahar ^{[1
]}

Vavasis, Stephen ^{[1
]}

机构：

[1] Univ Waterloo, Dept Combinator & Optimizat, 200 Univ Ave W, Waterloo, ON N2L 3G1, Canada

来源：

SIAM JOURNAL ON OPTIMIZATION | 2017年 / 27卷 / 02期

基金：

加拿大自然科学与工程研究理事会;

关键词：

proximal methods; quasi-Newton methods; sparse recovery; basis pursuit denoising problem; l(1)-regularized least squares problem; convex optimization; minimization of composite functions; PROJECTED GRADIENT METHODS; SIGNAL RECOVERY; ALGORITHM; SPARSE; RECONSTRUCTION; OPTIMIZATION; L(1)-MINIMIZATION; MINIMIZATION; SHRINKAGE;

D O I：

10.1137/140966587

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We present a proximal quasi-Newton method in which the approximation of the Hessian has the special format of "identity minus rank one" (IMRO) in each iteration. The proposed structure enables us to effectively recover the proximal point. The algorithm is applied to l(1)-regularized least squares problems arising in many applications including sparse recovery in compressive sensing, machine learning, and statistics. Our numerical experiment suggests that the proposed technique competes favorably with other state-of-the-art solvers for this class of problems. We also provide a complexity analysis for variants of IMRO, showing that it matches known best bounds.

引用

页码：583 / 615

页数：33

共 50 条

[1] Proximal Quasi-Newton for Computationally Intensive l1-regularized M-estimators
Zhong, Kai
Yen, Ian E. H.
Dhillon, Inderjit S.
Ravikumar, Pradeep
[J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 27 (NIPS 2014), 2014, 27
[2] Global convergence of a regularized factorized quasi-Newton method for nonlinear least squares problems
Zhou, Weijun
Zhang, Li
[J]. COMPUTATIONAL & APPLIED MATHEMATICS, 2010, 29 (02): : 195 - 214
[3] CONVERGENCE OF AN INERTIAL PROXIMAL METHOD FOR L1-REGULARIZED LEAST-SQUARES
Johnstone, Patrick R.
Moulin, Pierre
[J]. 2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 2015, : 3566 - 3570
[4] Convergence of Common Proximal Methods for l1-Regularized Least Squares
Tao, Shaozhe
Boley, Daniel
Zhang, Shuzhong
[J]. PROCEEDINGS OF THE TWENTY-FOURTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE (IJCAI), 2015, : 3849 - 3855
[5] An active-set proximal quasi-Newton algorithm for l1-regularized minimization over a sphere constraint
Shen, Chungen
Mi, Ling
Zhang, Lei-Hong
[J]. OPTIMIZATION, 2022, 71 (16) : 4623 - 4664
[6] REGULARIZED QUASI-NEWTON METHOD FOR INVERSE SCATTERING PROBLEMS
GUTMAN, S
KLIBANOV, M
[J]. MATHEMATICAL AND COMPUTER MODELLING, 1993, 18 (01) : 5 - 31
[7] On the similarities between the quasi-Newton least squares method and GMRes
Haelterman, R.
Lauwens, B.
Van Utterbeeck, F.
Bruyninckx, H.
Vierendeels, J.
[J]. JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2015, 273 : 25 - 28
[8] Wiring Diagnostics Via l1-Regularized Least Squares
Schuet, Stefan
[J]. IEEE SENSORS JOURNAL, 2010, 10 (07) : 1218 - 1225
[9] FACTORIZED QUASI-NEWTON METHODS FOR NONLINEAR LEAST-SQUARES PROBLEMS
YABE, H
TAKAHASHI, T
[J]. MATHEMATICAL PROGRAMMING, 1991, 51 (01) : 75 - 100
[10] A CLASS OF FACTORIZED QUASI-NEWTON METHODS FOR NONLINEAR LEAST SQUARES PROBLEMS
C.X. Xu
X.F. Ma
M.Y. Kong(Department of Mathematics
[J]. Journal of Computational Mathematics, 1996, (02) : 143 - 158

← 1 2 3 4 5 →