Convergence of the Wake-Sleep algorithm

被引：0
|
作者：
Ikeda, S [1 ]
Amari, S [1 ]
Nakahara, H [1 ]
机构：
[1] PRESTO, JST, Wako, Saitama 3510198, Japan
来源：
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 11 | 1999年 / 11卷
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The W-S (Wake-Sleep) algorithm is a simple learning rule for the models with hidden variables. It is shown that this algorithm can be applied to a factor analysis model which is a linear version of the Helmholtz machine. But even for a factor analysis model, the general convergence is not proved theoretically. In this article, we describe the geometrical understanding of the W-S algorithm in contrast with the EM (Expectation-Maximization) algorithm and the em algorithm. As the result, we prove the convergence of the W-S algorithm for the factor analysis model. We also show the condition for the convergence in general models.
引用
收藏
页码：239 / 245
页数：7
相关论文
共 50 条
  • [1] Dynamical analysis of the wake-sleep algorithm
    Shimasaki, S
    Kabashima, Y
    ELECTRONICS AND COMMUNICATIONS IN JAPAN PART III-FUNDAMENTAL ELECTRONIC SCIENCE, 2002, 85 (01): : 41 - 49
  • [2] 'Wake-sleep' algorithm for unsupervised neural networks
    Hinton, G.E.
    Dayan, P.
    Frey, B.J.
    Neal, R.M.
    Science, 1995, 268 (5214):
  • [3] THE WAKE-SLEEP ALGORITHM FOR UNSUPERVISED NEURAL NETWORKS
    HINTON, GE
    DAYAN, P
    FREY, BJ
    NEAL, RM
    SCIENCE, 1995, 268 (5214) : 1158 - 1161
  • [4] Wake-sleep PCA
    Choi, Seungjin
    2007 IEEE INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-6, 2007, : 2431 - 2434
  • [5] Wake-sleep circuitry: an overview
    Saper, Clifford B.
    Fuller, Patrick M.
    CURRENT OPINION IN NEUROBIOLOGY, 2017, 44 : 186 - 192
  • [6] Models of MT and MST areas using wake-sleep algorithm
    Katayama, K
    Ando, M
    Horiguchi, T
    NEURAL NETWORKS, 2004, 17 (03) : 339 - 351
  • [7] Wake-Sleep Consolidated Learning
    Sorrenti, Amelia
    Bellitto, Giovanni
    Salanitri, Federica Proietto
    Pennisi, Matteo
    Palazzo, Simone
    Spampinato, Concetto
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2024,
  • [8] Does the wake-sleep algorithm produce good density estimators?
    Frey, BJ
    Hinton, GE
    Dayan, P
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 8: PROCEEDINGS OF THE 1995 CONFERENCE, 1996, 8 : 661 - 667
  • [9] Natural Reweighted Wake-Sleep
    Varady, Csongor
    Volpi, Riccardo
    Malago, Luigi
    Ay, Nihat
    NEURAL NETWORKS, 2022, 155 : 574 - 591
  • [10] Affect Across the Wake-Sleep Cycle
    Sikka, Pilleriin
    Gross, James J.
    AFFECTIVE SCIENCE, 2023, 4 (03) : 563 - 569
12345