ACCELERATED STOCHASTIC APPROXIMATION

被引：36
|
作者：
Delyon, Bernard [1 ]
Juditsky, Anatoli [1 ]
机构：
[1] Inst Natl Rech Informat & Automat, Inst Rech Informat & Syst Aleatoires, F-35042 Rennes, France
来源：
SIAM JOURNAL ON OPTIMIZATION | 1993年 / 3卷 / 04期
关键词：
stochastic approximation; accelerated algorithms; optimal algorithms;
D O I：
10.1137/0803045
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
A technique to accelerate convergence of stochastic approximation algorithms is studied. It is based on Kesten's idea of equalization of the gain coefficient for the Robbins-Monro algorithm. Convergence with probability is proved for the multidimensional analog of the Kesten accelerated stochastic approximation algorithm. Asymptotic normality of the delivered estimates is also shown. Results of numerical simulations are presented that demonstrate the efficiency of the acceleration procedure.
引用
收藏
页码：868 / 881
页数：14
相关论文
共 50 条
  • [31] MULTIEXTREMAL STOCHASTIC APPROXIMATION
    VAYSBORD, EM
    YUDIN, DB
    ENGINEERING CYBERNETICS, 1968, (05): : 1 - &
  • [32] Stochastic continuity and approximation
    Brown, L
    Schreiber, BM
    STUDIA MATHEMATICA, 1996, 121 (01) : 15 - 33
  • [33] Formal Comparison of Simultaneous Perturbation Stochastic Approximation and Random Direction Stochastic Approximation
    Peng, Ducheng
    Chen, Yiwen
    Spall, James C.
    2023 AMERICAN CONTROL CONFERENCE, ACC, 2023, : 744 - 749
  • [34] Accelerated randomized stochastic optimization
    Dippon, J
    ANNALS OF STATISTICS, 2003, 31 (04): : 1260 - 1281
  • [35] Accelerated Stochastic Power Iteration
    Xu, Peng
    He, Bryan
    De Sa, Christopher
    Mitliagkas, Ioannis
    Re, Christopher
    INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 84, 2018, 84
  • [36] Accelerated Stochastic Variational Inference
    Hu, Pingbo
    Weng, Yang
    2019 IEEE INTL CONF ON PARALLEL & DISTRIBUTED PROCESSING WITH APPLICATIONS, BIG DATA & CLOUD COMPUTING, SUSTAINABLE COMPUTING & COMMUNICATIONS, SOCIAL COMPUTING & NETWORKING (ISPA/BDCLOUD/SOCIALCOM/SUSTAINCOM 2019), 2019, : 1275 - 1282
  • [37] Combining the Stochastic Counterpart and Stochastic Approximation Methods
    Jean-Pierre Dussault
    Donald Labrecque
    Pierre L'Ecuyer
    Reuven Y. Rubinstein
    Discrete Event Dynamic Systems, 1997, 7 : 5 - 28
  • [38] A STOCHASTIC APPROXIMATION ALGORITHM FOR STOCHASTIC SEMIDEFINITE PROGRAMMING
    Gaujal, Bruno
    Mertikopoulos, Panayotis
    PROBABILITY IN THE ENGINEERING AND INFORMATIONAL SCIENCES, 2016, 30 (03) : 431 - 454
  • [39] Stochastic Hybrid Approximation Algorithm for Stochastic Programs
    Shi, Ning
    Lai, K. K.
    PROCEEDINGS OF 2008 INTERNATIONAL SYMPOSIUM ON APPLIED COMPUTING AND COMPUTATIONAL SCIENCES: ADVANCES IN APPLIED COMPUTING AND COMPUTATIONAL SCIENCES, 2008, : 18 - +
  • [40] Combining the stochastic counterpart and stochastic approximation methods
    Dussault, JP
    Labrecque, D
    LEcuyer, P
    Rubinstein, RY
    DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS, 1997, 7 (01): : 5 - 28
12345