MULTIPLICATIVE PARAMETERS IN GRADIENT DESCENT METHODS

被引：2

作者：

Stanimirovic, Predrag ^{[1
]}

Miladinovic, Marko ^{[1
]}

Djordjevic, Snezana ^{[1
]}

机构：

[1] Univ Nis, Dept Math, Fac Sci & Math, Nish 18000, Serbia

来源：

FILOMAT | 2009年 / 23卷 / 03期

关键词：

Line search; gradient descent methods; Newton method; backtracking line search; convergence rate; BARZILAI;

D O I：

10.2298/FIL0903023S

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We introduced an algorithm for unconstrained optimization based on the reduction of the modified Newton method with line search into a gradient descent method. Main idea used in the algorithm construction is approximation of Hessian by a diagonal matrix. The step length calculation algorithm is based on the Taylor's development in two successive iterative points and the backtracking line search procedure.

引用

页码：23 / 36

页数：14

共 50 条

[21] ON THE ITERATION COMPLEXITY OF CYCLIC COORDINATE GRADIENT DESCENT METHODS
Yun, Sangwoon
SIAM JOURNAL ON OPTIMIZATION, 2014, 24 (03) : 1567 - 1580
[22] Steepest descent and conjugate gradient methods with variable preconditioning
Knyazev, Andrew V.
Lashuk, Ilya
SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS, 2007, 29 (04) : 1267 - 1280
[23] Iterative Gradient Descent Methods for Solving Linear Equations
Samokhin, A. B.
Samokhina, A. S.
Sklyar, A. Ya
Shestopalov, Yu, V
COMPUTATIONAL MATHEMATICS AND MATHEMATICAL PHYSICS, 2019, 59 (08) : 1267 - 1274
[24] Multiobjective Sequence Design via Gradient Descent Methods
Baden, John Michael
O'Donnell, Brian
Schmieder, Lance
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS, 2018, 54 (03) : 1237 - 1252
[25] Learning to Learn without Gradient Descent by Gradient Descent
Chen, Yutian
Hoffman, Matthew W.
Colmenarejo, Sergio Gomez
Denil, Misha
Lillicrap, Timothy P.
Botvinick, Matt
de Freitas, Nando
INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 70, 2017, 70
[26] Updating simulation model parameters using stochastic gradient descent
Ali, Mostafa
AbouRizk, Simaan
AUTOMATION IN CONSTRUCTION, 2024, 166
[27] Multiplicative interior gradient methods for minimization over the nonnegative orthant
Iusem, AN
Svaiter, BF
Teboulle, M
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 1996, 34 (01) : 389 - 406
[28] A multiplicative gradient descent search algorithm for user preference retrieval and its application to web search
Meng, XN
Chen, ZX
Spink, A
ITCC 2003: INTERNATIONAL CONFERENCE ON INFORMATION TECHNOLOGY: COMPUTERS AND COMMUNICATIONS, PROCEEDINGS, 2003, : 150 - 154
[29] Some sufficient descent conjugate gradient methods and their global convergence
Min Li
Aiping Qu
Computational and Applied Mathematics, 2014, 33 : 333 - 347
[30] Gradient-descent methods for parameter estimation in chaotic systems
Mariño, IP
Míguez, J
ISPA 2005: PROCEEDINGS OF THE 4TH INTERNATIONAL SYMPOSIUM ON IMAGE AND SIGNAL PROCESSING AND ANALYSIS, 2005, : 440 - 445

← 1 2 3 4 5 →