Adaptive stochastic approximation algorithm

被引：4

作者：

Kresoja, Milena ^{[1
]}

Luzanin, Zorana ^{[1
]}

Stojkovska, Irena ^{[2
]}

机构：

[1] Univ Novi Sad, Fac Sci, Dept Math & Informat, Trg Dositeja Obradovica 4, Novi Sad 21000, Serbia

[2] Ss Cyril & Methodius Univ, Fac Nat Sci & Math, Dept Math, Arhimedova 3, Skopje 1000, Macedonia

来源：

NUMERICAL ALGORITHMS | 2017年 / 76卷 / 04期

关键词：

Unconstrained optimization; Stochastic optimization; Stochastic approximation; Noisy function; Adaptive step size; Gradient method; Descent direction; OPTIMIZATION METHODS; GRADIENT;

D O I：

10.1007/s11075-017-0290-4

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure convergence is established. The case when negative gradient is chosen as a search direction is also considered. The algorithm is tested on a set of standard test problems. Numerical results show good performance and verify efficiency of the algorithm compared to some of existing algorithms with adaptive step sizes.

引用

页码：917 / 937

页数：21

共 50 条

[31] ON BATHER STOCHASTIC-APPROXIMATION ALGORITHM
SCHWABE, R
KYBERNETIKA, 1994, 30 (03) : 301 - 306
[32] Stochastic State Estimation Using a New Adaptive Sparse Pseudospectral Approximation Method: Model and Algorithm
Lin J.
Shen D.
Tao X.
Liu Y.
Zhongguo Dianji Gongcheng Xuebao/Proceedings of the Chinese Society of Electrical Engineering, 2019, 39 (01): : 192 - 203
[33] OPTIMIZING ADAPTIVE IMPORTANCE SAMPLING BY STOCHASTIC APPROXIMATION
Kawai, Reiichiro
SIAM JOURNAL ON SCIENTIFIC COMPUTING, 2018, 40 (04): : A2774 - A2800
[34] ADAPTIVE SIGNAL PROCESSING THROUGH STOCHASTIC APPROXIMATION
WANG, RJ
TREITEL, S
GEOPHYSICS, 1972, 37 (02) : 389 - &
[35] ADAPTIVE STOCHASTIC APPROXIMATION FOR MEASUREMENT WITH CORRELATED NOISE
GAO Aijun (Institute of Mathematical Sciences
Systems Science and Mathematical Sciences, 1992, (03) : 213 - 226
[36] Adaptive weak approximation of stochastic differential equations
Szepessy, A
Tempone, R
Zouraris, GE
COMMUNICATIONS ON PURE AND APPLIED MATHEMATICS, 2001, 54 (10) : 1169 - 1214
[37] EXTENSIONS OF KESTENS ADAPTIVE STOCHASTIC APPROXIMATION METHOD
KUSHNER, HJ
GAVIN, T
ANNALS OF STATISTICS, 1973, 1 (05): : 851 - 861
[38] ADAPTIVE KALMAN FILTERING USING STOCHASTIC APPROXIMATION
SINHA, NK
ELECTRONICS LETTERS, 1973, 9 (8-9) : 177 - 178
[39] Adaptive stochastic approximation by the simultaneous perturbation method
Spall, JC
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2000, 45 (10) : 1839 - 1853
[40] Adaptive stochastic approximation by the simultaneous perturbation method
Spall, JC
PROCEEDINGS OF THE 37TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-4, 1998, : 3872 - 3879

← 1 2 3 4 5 →