Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics

被引：0

作者：

Lovas, A. ^{[1
,2
]}

Rasonyi, M. ^{[1
,3
]}

机构：

[1] Alfred Renyi Inst Math, Budapest, Hungary

[2] Budapest Univ Technol & Econ, Budapest, Hungary

[3] Eotvos Lorand Univ, Budapest, Hungary

来源：

APPLIED MATHEMATICS AND OPTIMIZATION | 2023年 / 88卷 / 03期

关键词：

Stochastic gradient descent; Online learning; Functional central limit theorem; Mixing; Markov chains in random environments; DEPENDENT DATA STREAMS; APPROXIMATION;

D O I：

10.1007/s00245-023-10052-y

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a Markov chain in a random environment, which complicates themathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.

引用

页数：22

共 50 条

[1] Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics
A. Lovas
M. Rásonyi
[J]. Applied Mathematics & Optimization, 2023, 88
[2] ON STRONG LAW OF LARGE NUMBERS AND CENTRAL LIMIT THEOREM FOR MARTINGALES
CSORGO, M
[J]. TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY, 1968, 131 (01) : 259 - &
[3] ON STRONG LAW OF LARGE NUMBERS AND CENTRAL LIMIT THEOREM FOR MARTINGALES
CSORGO, M
[J]. CANADIAN MATHEMATICAL BULLETIN, 1966, 9 (03): : 386 - &
[4] A spatial stochastic epidemic model: law of large numbers and central limit theorem
Bowong, S.
Emakoua, A.
Pardoux, E.
[J]. STOCHASTICS AND PARTIAL DIFFERENTIAL EQUATIONS-ANALYSIS AND COMPUTATIONS, 2023, 11 (01): : 31 - 105
[5] A spatial stochastic epidemic model: law of large numbers and central limit theorem
S. Bowong
A. Emakoua
E. Pardoux
[J]. Stochastics and Partial Differential Equations: Analysis and Computations, 2023, 11 : 31 - 105
[6] CENTRAL LIMIT-THEOREM AND THE LAW OF LARGE NUMBERS
KRUGLOV, VM
[J]. MATHEMATICAL NOTES, 1981, 30 (3-4) : 786 - 791
[7] CENTRAL LIMIT-THEOREM AND LAW OF LARGE NUMBERS IN THE MEAN
KRUGLOV, VM
[J]. THEORY OF PROBABILITY AND ITS APPLICATIONS, 1982, 26 (04) : 813 - 815
[8] Counterexamples to a Central Limit Theorem and a Weak Law of Large Numbers for capacities
Teran, Pedro
[J]. STATISTICS & PROBABILITY LETTERS, 2015, 96 : 185 - 189
[9] Law of large numbers and central limit theorem for ergodic quantum processes
Pathirana, Lubashan
Schenker, Jeffrey
[J]. JOURNAL OF MATHEMATICAL PHYSICS, 2023, 64 (08)
[10] Law of large numbers and central limit theorem under nonlinear expectations
Peng, Shige
[J]. PROBABILITY UNCERTAINTY AND QUANTITATIVE RISK, 2019, 4

← 1 2 3 4 5 →