Stochastic gradient algorithm with random truncations

被引:5
|
作者
Tadic, V
机构
[1] Mihajlo Pupin Inst, Belgrade, Yugoslavia
关键词
gradient methods; stochastic optimization; stochastic approximation; uniform mixing;
D O I
10.1016/S0377-2217(96)00397-9
中图分类号
C93 [管理学];
学科分类号
12 ; 1201 ; 1202 ; 120202 ;
摘要
Let f : R-d x R-d' --> R be a Borel-measurable function which satisfies integral(Rd')\f(theta,x)\q(0)(dx) < infinity, For All theta is an element of R-d, where q(0)(.) is a probability measure on (R-d', B-d'). The problem of minimization of the function f(0)(theta) = integral(Rd') f(theta,x)q(0)(dx), theta is an element of R-d, is considered for the case when the probability measure q(0)(.), is unknown, but a realization of a non-stationary random process {X-n}(n greater than or equal to 1) whose single probability measures in a certain sense tend to q(0)(.), is available. The random process {X-n}(n greater than or equal to 1) is defined on a common probability space, R-d'-valued, correlated and satisfies certain uniform mixing conditions. The function f(.,.) is completely known. A stochastic gradient algorithm with random truncations is used for the minimization of f(0)(.), and its almost sure convergence is proved. (C) 1997 Elsevier Science B.V.
引用
收藏
页码:261 / 284
页数:24
相关论文
共 50 条
  • [21] A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning
    Shim, Duk-Sun
    Shim, Joseph
    INTERNATIONAL JOURNAL OF CONTROL AUTOMATION AND SYSTEMS, 2023, 21 (11) : 3825 - 3831
  • [22] A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning
    Duk-Sun Shim
    Joseph Shim
    International Journal of Control, Automation and Systems, 2023, 21 : 3825 - 3831
  • [23] On the eigenvalues of truncations of random unitary matrices
    Meckes, Elizabeth
    Stewart, Kathryn
    ELECTRONIC COMMUNICATIONS IN PROBABILITY, 2019, 24
  • [24] Why random reshuffling beats stochastic gradient descent
    M. Gürbüzbalaban
    A. Ozdaglar
    P. A. Parrilo
    Mathematical Programming, 2021, 186 : 49 - 84
  • [25] Private weighted random walk stochastic gradient descent
    Ayache G.
    El Rouayheb S.
    IEEE Journal on Selected Areas in Information Theory, 2021, 2 (01): : 452 - 463
  • [26] Why random reshuffling beats stochastic gradient descent
    Gurbuzbalaban, M.
    Ozdaglar, A.
    Parrilo, P. A.
    MATHEMATICAL PROGRAMMING, 2021, 186 (1-2) : 49 - 84
  • [27] Stochastic Proximal Gradient Consensus Over Random Networks
    Hong, Mingyi
    Chang, Tsung-Hui
    IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2017, 65 (11) : 2933 - 2948
  • [28] RANDOM COEFFICIENT MODEL TO ESTIMATE A STOCHASTIC DENSITY GRADIENT
    KAU, JB
    LEE, CF
    REGIONAL SCIENCE AND URBAN ECONOMICS, 1977, 7 (1-2) : 169 - 177
  • [29] Stochastic gradient estimation strategies for Markov random fields
    Younes, L
    BAYESIAN INFERENCE FOR INVERSE PROBLEMS, 1998, 3459 : 315 - 325
  • [30] Modified Stochastic Gradient Algorithm for Additive Systems
    Wang, Fan
    INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND COMMUNICATION ENGINEERING (CSCE 2015), 2015, : 679 - 684