Stochastic Gradient Langevin Dynamics with Variance Reduction

被引：2
|
作者：
Huang, Zhishen [1 ]
Becker, Stephen [2 ]
机构：
[1] Michigan State Univ, Dept Computat Math, E Lansing, MI 48824 USA
[2] Univ Colorado, Dept Appl Math, Boulder, CO 80309 USA
来源：
2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) | 2021年
基金：
美国国家科学基金会;
关键词：
GLOBAL OPTIMIZATION; CONVERGENCE; DIFFUSION; HASTINGS;
D O I：
10.1109/IJCNN52387.2021.9533646
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Stochastic gradient Langevin dynamics (SGLD) has gained the attention of optimization researchers due to its global optimization properties. This paper proves an improved convergence property to local minimizers of nonconvex objective functions using SGLD accelerated by variance reductions. Moreover, we prove an ergodicity property of the SGLD scheme, which gives insights on its potential to find global minimizers of nonconvex objectives.
引用
收藏
页数：8
相关论文
共 50 条
  • [21] Sampling from Non-Log-Concave Distributions via Stochastic Variance-Reduced Gradient Langevin Dynamics
    Zou, Difan
    Xu, Pan
    Gu, Quanquan
    [J]. 22ND INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 89, 2019, 89
  • [22] Distributed and asynchronous Stochastic Gradient Descent with variance reduction
    Ming, Yuewei
    Zhao, Yawei
    Wu, Chengkun
    Li, Kuan
    Yin, Jianping
    [J]. NEUROCOMPUTING, 2018, 281 : 27 - 36
  • [23] On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
    Chatterji, Niladri S.
    Flammarion, Nicolas
    Ma, Yi-An
    Bartlett, Peter L.
    Jordan, Michael I.
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 80, 2018, 80
  • [24] A Stochastic Composite Gradient Method with Incremental Variance Reduction
    Zhang, Junyu
    Xiao, Lin
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), 2019, 32
  • [25] PROXIMAL STOCHASTIC GRADIENT METHOD WITH PROGRESSIVE VARIANCE REDUCTION
    Xiao, Lin
    Zhang, Tong
    [J]. SIAM JOURNAL ON OPTIMIZATION, 2014, 24 (04) : 2057 - 2075
  • [26] Stochastic Gradient and Langevin Processes
    Cheng, Xiang
    Yin, Dong
    Bartlett, Peter
    Jordan, Michael
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 119, 2020, 119
  • [27] Stochastic Gradient and Langevin Processes
    Cheng, Xiang
    Yin, Dong
    Bartlett, Peter
    Jordan, Michael
    [J]. 25TH AMERICAS CONFERENCE ON INFORMATION SYSTEMS (AMCIS 2019), 2019,
  • [28] On stochastic gradient Langevin dynamics with dependent data streams in the logconcave case
    Barkhagen, M.
    Chau, N. H.
    Moulines, E.
    Rasonyi, M.
    Sabanis, S.
    Zhang, Y.
    [J]. BERNOULLI, 2021, 27 (01) : 1 - 33
  • [29] Nonasymptotic Estimation of Risk Measures Using Stochastic Gradient Langevin Dynamics
    Chu, Jiarui
    Tangpi, Ludovic
    [J]. SIAM JOURNAL ON FINANCIAL MATHEMATICS, 2024, 15 (02): : 503 - 536
  • [30] HYBRID DETERMINISTIC-STOCHASTIC GRADIENT LANGEVIN DYNAMICS FOR BAYESIAN LEARNING
    He, Qi
    Xin, Jack
    [J]. COMMUNICATIONS IN INFORMATION AND SYSTEMS, 2012, 12 (03) : 221 - 232
12345