MULTIPLICATIVE PARAMETERS IN GRADIENT DESCENT METHODS

被引：2

作者：

Stanimirovic, Predrag ^{[1
]}

Miladinovic, Marko ^{[1
]}

Djordjevic, Snezana ^{[1
]}

机构：

[1] Univ Nis, Dept Math, Fac Sci & Math, Nish 18000, Serbia

来源：

FILOMAT | 2009年 / 23卷 / 03期

关键词：

Line search; gradient descent methods; Newton method; backtracking line search; convergence rate; BARZILAI;

D O I：

10.2298/FIL0903023S

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We introduced an algorithm for unconstrained optimization based on the reduction of the modified Newton method with line search into a gradient descent method. Main idea used in the algorithm construction is approximation of Hessian by a diagonal matrix. The step length calculation algorithm is based on the Taylor's development in two successive iterative points and the backtracking line search procedure.

引用

页码：23 / 36

页数：14

共 50 条

[11] A Class of Descent Nonlinear Conjugate Gradient Methods
Ying, Tao
2013 FOURTH INTERNATIONAL CONFERENCE ON DIGITAL MANUFACTURING AND AUTOMATION (ICDMA), 2013, : 14 - 16
[12] STATISTICAL INFERENCE FOR MODEL PARAMETERS IN STOCHASTIC GRADIENT DESCENT
Chen, Xi
Lee, Jason D.
Tong, Xin T.
Zhang, Yichen
ANNALS OF STATISTICS, 2020, 48 (01): : 251 - 273
[13] Optimization of Gradient Descent Parameters in Attitude Estimation Algorithms
Sever, Karla
Golusin, Leonardo Max
Loncar, Josip
SENSORS, 2023, 23 (04)
[14] Learning to learn by gradient descent by gradient descent
Andrychowicz, Marcin
Denil, Misha
Colmenarejo, Sergio Gomez
Hoffman, Matthew W.
Pfau, David
Schaul, Tom
Shillingford, Brendan
de Freitas, Nando
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 29 (NIPS 2016), 2016, 29
[15] Nonsymmetric preconditioning for conjugate gradient and steepest descent methods
Bouwmeester, Henricus
Dougherty, Andrew
Knyazev, Andrew V.
INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE, ICCS 2015 COMPUTATIONAL SCIENCE AT THE GATES OF NATURE, 2015, 51 : 276 - 285
[16] Analysis of Weight Initialization Methods for Gradient Descent with momentum
Masood, Sarfaraz
Doja, M. N.
Chandra, Pravin
2015 INTERNATIONAL CONFERENCE ON SOFT COMPUTING TECHNIQUES AND IMPLEMENTATIONS (ICSCTI), 2015,
[17] Privacy-Preserving Gradient-Descent Methods
Han, Shuguo
Ng, Wee Keong
Wan, Li
Lee, Vincent C. S.
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2010, 22 (06) : 884 - 899
[18] Iterative Gradient Descent Methods for Solving Linear Equations
A. B. Samokhin
A. S. Samokhina
A. Ya. Sklyar
Yu. V. Shestopalov
Computational Mathematics and Mathematical Physics, 2019, 59 : 1267 - 1274
[19] Analysis of Gradient Descent Methods With Nondiminishing Bounded Errors
Ramaswamy, Arunselvan
Bhatnagar, Shalabh
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2018, 63 (05) : 1465 - 1471
[20] Two descent hybrid conjugate gradient methods for optimization
Zhang, Li
Zhou, Weijun
JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2008, 216 (01) : 251 - 264

← 1 2 3 4 5 →