Almost sure convergence of stochastic gradient processes with matrix step sizes

被引：6

作者：

Monnez, JM ^{[1
]}

机构：

[1] Univ Henri Poincare, IECN Lab Math, F-54506 Vandoeuvre Les Nancy, France

来源：

STATISTICS & PROBABILITY LETTERS | 2006年 / 76卷 / 05期

关键词：

stochastic approximation; stochastic gradient;

D O I：

10.1016/j.spl.2005.09.014

中图分类号：

O21 [概率论与数理统计]; C8 [统计学];

学科分类号：

020208 ; 070103 ; 0714 ;

摘要：

We consider a stochastic gradient process, which is a special case of stochastic approximation process, where the positive real step size a(n) is replaced by a random matrix A(n): Xn+1 = X-n - A(n)del g(X-n) - A(n)V(n). We give two theorems of almost sure convergence in the case where the equation del g = 0 has a set of solutions. (C) 2005 Elsevier B.V. All rights reserved.

引用

页码：531 / 536

页数：6

共 50 条

[1] Further results on almost sure convergence of the stochastic gradient algorithm
Radenkovic, MS
[J]. INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING, 1999, 13 (03) : 161 - 181
[2] Almost sure convergence rates of stochastic proximal gradient descent algorithm
Liang, Yuqing
Xu, Dongpo
[J]. OPTIMIZATION, 2024, 73 (08) : 2413 - 2446
[3] Almost Sure Uniform Convergence of Stochastic Processes in the Dual of a Nuclear Space
C. A. Fonseca-Mora
[J]. Journal of Theoretical Probability, 2023, 36 : 2564 - 2589
[4] Almost Sure Uniform Convergence of Stochastic Processes in the Dual of a Nuclear Space
Fonseca-Mora, C. A.
[J]. JOURNAL OF THEORETICAL PROBABILITY, 2023, 36 (01) : 1 - 26
[5] On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems
Mertikopoulos, Panayotis
Hallak, Nadav
Kavis, Ali
Cevher, Volkan
[J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
[6] On almost sure convergence results in stochastic calculus
Karandikar, Rajeeva L.
[J]. IN MEMORIAM PAUL-ANDRE MEYER: SEMINAIRE DE PROBABILITIES XXXIX, 2006, 1874 : 137 - 147
[7] On almost sure convergence for sums of stochastic sequence
Wang, Zhong-zhi
Dong, Yun
Ding, Fangqing
[J]. COMMUNICATIONS IN STATISTICS-THEORY AND METHODS, 2019, 48 (14) : 3609 - 3621
[8] ALMOST SURE CONVERGENCE OF A CLASS OF STOCHASTIC ALGORITHMS
BISCARAT, JC
[J]. STOCHASTIC PROCESSES AND THEIR APPLICATIONS, 1994, 50 (01) : 83 - 99
[9] ALMOST SURE CONVERGENCE OF DELAYED RENEWAL PROCESSES
STEINEBACH, J
[J]. JOURNAL OF THE LONDON MATHEMATICAL SOCIETY-SECOND SERIES, 1987, 36 : 569 - 576
[10] ALMOST SURE CONVERGENCE OF BRANCHING-PROCESSES
COHN, H
[J]. ZEITSCHRIFT FUR WAHRSCHEINLICHKEITSTHEORIE UND VERWANDTE GEBIETE, 1977, 38 (01): : 73 - 81

← 1 2 3 4 5 →