Stochastic Gradient Langevin Dynamics with Variance Reduction

被引：2
|
作者：
Huang, Zhishen [1 ]
Becker, Stephen [2 ]
机构：
[1] Michigan State Univ, Dept Computat Math, E Lansing, MI 48824 USA
[2] Univ Colorado, Dept Appl Math, Boulder, CO 80309 USA
来源：
2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) | 2021年
基金：
美国国家科学基金会;
关键词：
GLOBAL OPTIMIZATION; CONVERGENCE; DIFFUSION; HASTINGS;
D O I：
10.1109/IJCNN52387.2021.9533646
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Stochastic gradient Langevin dynamics (SGLD) has gained the attention of optimization researchers due to its global optimization properties. This paper proves an improved convergence property to local minimizers of nonconvex objective functions using SGLD accelerated by variance reductions. Moreover, we prove an ergodicity property of the SGLD scheme, which gives insights on its potential to find global minimizers of nonconvex objectives.
引用
收藏
页数：8
相关论文
共 50 条
  • [1] Variance Reduction in Stochastic Gradient Langevin Dynamics
    Dubey, Avinava
    Reddi, Sashank J.
    Poczos, Barnabas
    Smola, Alexander J.
    Xing, Eric P.
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 29 (NIPS 2016), 2016, 29
  • [2] Subsampled Stochastic Variance-Reduced Gradient Langevin Dynamics
    Zou, Difan
    Xu, Pan
    Gu, Quanquan
    [J]. UNCERTAINTY IN ARTIFICIAL INTELLIGENCE, 2018, : 508 - 518
  • [3] Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization
    Kinoshita, Yuri
    Suzuki, Taiji
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [4] Approximation to Stochastic Variance Reduced Gradient Langevin Dynamics by Stochastic Delay Differential Equations
    Chen, Peng
    Lu, Jianya
    Xu, Lihu
    [J]. APPLIED MATHEMATICS AND OPTIMIZATION, 2022, 85 (02):
  • [5] Approximation to Stochastic Variance Reduced Gradient Langevin Dynamics by Stochastic Delay Differential Equations
    Peng Chen
    Jianya Lu
    Lihu Xu
    [J]. Applied Mathematics & Optimization, 2022, 85
  • [6] Exploration of the (Non-)Asymptotic Bias and Variance of Stochastic Gradient Langevin Dynamics
    Vollmer, Sebastian J.
    Zygalakis, Konstantinos C.
    Teh, Yee Whye
    [J]. JOURNAL OF MACHINE LEARNING RESEARCH, 2016, 17 : 1 - 45
  • [7] Convergence of Mean-field Langevin dynamics: Time-space discretization, stochastic gradient, and variance reduction
    Suzuki, Taiji
    Wu, Denny
    Nitanda, Atsushi
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [8] Consistency and Fluctuations For Stochastic Gradient Langevin Dynamics
    Teh, Yee Whye
    Thiery, Alexandre H.
    Vollmer, Sebastian J.
    [J]. JOURNAL OF MACHINE LEARNING RESEARCH, 2016, 17
  • [9] The promises and pitfalls of Stochastic Gradient Langevin Dynamics
    Brosse, Nicolas
    Moulines, Eric
    Durmus, Alain
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018), 2018, 31
  • [10] Consistency and fluctuations for stochastic gradient Langevin dynamics
    [J]. 1600, Microtome Publishing (17):
12345