MULTIPLICATIVE PARAMETERS IN GRADIENT DESCENT METHODS

被引：2
|
作者：
Stanimirovic, Predrag [1 ]
Miladinovic, Marko [1 ]
Djordjevic, Snezana [1 ]
机构：
[1] Univ Nis, Dept Math, Fac Sci & Math, Nish 18000, Serbia
来源：
FILOMAT | 2009年 / 23卷 / 03期
关键词：
Line search; gradient descent methods; Newton method; backtracking line search; convergence rate; BARZILAI;
D O I：
10.2298/FIL0903023S
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
We introduced an algorithm for unconstrained optimization based on the reduction of the modified Newton method with line search into a gradient descent method. Main idea used in the algorithm construction is approximation of Hessian by a diagonal matrix. The step length calculation algorithm is based on the Taylor's development in two successive iterative points and the backtracking line search procedure.
引用
收藏
页码：23 / 36
页数：14
相关论文
共 50 条
  • [1] Two modifications of the method of the multiplicative parameters in descent gradient methods
    Dordevic, Snezana S.
    APPLIED MATHEMATICS AND COMPUTATION, 2012, 218 (17) : 8672 - 8683
  • [2] A note on a multiplicative parameters gradient method
    Dordevic, Snezana S.
    APPLIED MATHEMATICS AND COMPUTATION, 2016, 283 : 90 - 107
  • [3] Sufficient Descent Riemannian Conjugate Gradient Methods
    Sakai, Hiroyuki
    Iiduka, Hideaki
    JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 2021, 190 (01) : 130 - 150
  • [4] Sufficient Descent Riemannian Conjugate Gradient Methods
    Hiroyuki Sakai
    Hideaki Iiduka
    Journal of Optimization Theory and Applications, 2021, 190 : 130 - 150
  • [5] Distributed pairwise algorithms with gradient descent methods
    Wang, Baobin
    Hu, Ting
    NEUROCOMPUTING, 2019, 333 : 364 - 373
  • [6] Accelerated gradient descent methods with line search
    Stanimirovic, Predrag S.
    Miladinovic, Marko B.
    NUMERICAL ALGORITHMS, 2010, 54 (04) : 503 - 520
  • [7] The Chaotic Nature of Faster Gradient Descent Methods
    Kees van den Doel
    Uri Ascher
    Journal of Scientific Computing, 2012, 51 : 560 - 581
  • [8] Privacy-Preservation for Gradient Descent Methods
    Wan, Li
    Han, Shuguo
    Ng, Wee Keong
    Lee, Vincent C. S.
    KDD-2007 PROCEEDINGS OF THE THIRTEENTH ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, 2007, : 775 - +
  • [9] The Chaotic Nature of Faster Gradient Descent Methods
    van den Doel, Kees
    Ascher, Uri
    JOURNAL OF SCIENTIFIC COMPUTING, 2012, 51 (03) : 560 - 581
  • [10] Accelerated gradient descent methods with line search
    Predrag S. Stanimirović
    Marko B. Miladinović
    Numerical Algorithms, 2010, 54 : 503 - 520
12345