ACCELERATED STOCHASTIC APPROXIMATION

被引：36
|
作者：
Delyon, Bernard [1 ]
Juditsky, Anatoli [1 ]
机构：
[1] Inst Natl Rech Informat & Automat, Inst Rech Informat & Syst Aleatoires, F-35042 Rennes, France
来源：
SIAM JOURNAL ON OPTIMIZATION | 1993年 / 3卷 / 04期
关键词：
stochastic approximation; accelerated algorithms; optimal algorithms;
D O I：
10.1137/0803045
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
A technique to accelerate convergence of stochastic approximation algorithms is studied. It is based on Kesten's idea of equalization of the gain coefficient for the Robbins-Monro algorithm. Convergence with probability is proved for the multidimensional analog of the Kesten accelerated stochastic approximation algorithm. Asymptotic normality of the delivered estimates is also shown. Results of numerical simulations are presented that demonstrate the efficiency of the acceleration procedure.
引用
收藏
页码：868 / 881
页数：14
相关论文
共 50 条
  • [1] ACCELERATED STOCHASTIC APPROXIMATION
    ZHULENEV, SV
    TEORIYA VEROYATNOSTEI I YEYE PRIMENIYA, 1975, 20 (02): : 403 - 411
  • [2] ACCELERATED STOCHASTIC-APPROXIMATION
    KESTEN, H
    ANNALS OF MATHEMATICAL STATISTICS, 1958, 29 (01): : 41 - 59
  • [3] Accelerated stochastic approximation with state-dependent noise
    Ilandarideva, Sasila
    Juditsky, Anatoli
    Lan, Guanghui
    Li, Tianjiao
    MATHEMATICAL PROGRAMMING, 2024,
  • [4] Recursive least-squares and accelerated convergence in stochastic approximation schemes
    Ljung, L
    INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING, 2001, 15 (02) : 169 - 178
  • [5] Momentum-based accelerated mirror descent stochastic approximation for robust topology optimization under stochastic loads
    Li, Weichen
    Zhang, Xiaojia Shelly
    INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING, 2021, 122 (17) : 4431 - 4457
  • [6] Accelerated Simultaneous Perturbation Stochastic Approximation for Tracking Under Unknown-but-Bounded Disturbances
    Erofeeva, Victoria
    Granichin, Oleg
    Tursunova, Munira
    Sergeenko, Anna
    Jiang, Yuming
    2022 AMERICAN CONTROL CONFERENCE, ACC, 2022, : 1582 - 1587
  • [7] ONLINE DICTIONARY LEARNING FROM BIG DATA USING ACCELERATED STOCHASTIC APPROXIMATION ALGORITHMS
    Slavakis, Konstantinos
    Giannakis, Georgios B.
    2014 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2014,
  • [8] Accelerated spectral approximation
    Alam, R
    Kulkarni, RP
    Limaye, BV
    MATHEMATICS OF COMPUTATION, 1998, 67 (224) : 1401 - 1422
  • [9] STOCHASTIC APPROXIMATION
    ROBBINS, H
    MONRO, S
    ANNALS OF MATHEMATICAL STATISTICS, 1951, 22 (02): : 316 - 316
  • [10] Stochastic approximation
    Lai, TL
    ANNALS OF STATISTICS, 2003, 31 (02): : 391 - 406
12345