Attractor dynamics in feedforward neural networks

被引:14
|
作者
Saul, LK
Jordan, MI
机构
[1] AT&T Labs Res, Florham Pk, NJ 07932 USA
[2] Univ Calif Berkeley, Berkeley, CA 94720 USA
关键词
D O I
10.1162/089976600300015385
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We study the probabilistic generative models parameterized by feedforward neural networks. An attractor dynamics for probabilistic inference in these models is derived from a mean field approximation for large, layered sigmoidal networks. Fixed points of the dynamics correspond to solutions of the mean field equations, which relate the statistics of each unit to those of its Markov blanket. We establish global convergence of the dynamics by providing a Lyapunov function and show that the dynamics generate the signals required for unsupervised learning. Our results for feed forward networks provide a counterpart to those of Cohen-Grossberg and Hopfield for symmetric networks.
引用
收藏
页码:1313 / 1335
页数:23
相关论文
共 50 条
  • [1] Exact dynamics in feedforward neural networks
    Wong, KYM
    [J]. EUROPHYSICS LETTERS, 1997, 38 (08): : 631 - 636
  • [2] Fast terminal attractor based backpropagation algorithm for feedforward neural networks
    Batbayar, Batsukh
    Yu, Xinghuo
    [J]. PROCEEDINGS OF THE 2007 INTERNATIONAL CONFERENCE ON INTELLIGENT SENSORS, SENSOR NETWORKS AND INFORMATION PROCESSING, 2007, : 521 - 525
  • [3] DYNAMICS OF DILUTED ATTRACTOR NEURAL NETWORKS WITH DELAYS
    NUTZEL, K
    KIEN, J
    BAUER, K
    ALTMAN, JS
    KREY, U
    [J]. BIOLOGICAL CYBERNETICS, 1994, 70 (06) : 553 - 561
  • [4] Internal sigmoid dynamics in feedforward neural networks
    Gustafson, K
    [J]. CONNECTION SCIENCE, 1998, 10 (01) : 43 - 73
  • [5] Topology and dynamics of attractor neural networks: The role of loopiness
    Zhang, Pan
    Chen, Yong
    [J]. PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS, 2008, 387 (16-17) : 4411 - 4416
  • [6] DYNAMICS OF FULLY CONNECTED ATTRACTOR NEURAL NETWORKS NEAR SATURATION
    COOLEN, ACC
    SHERRINGTON, D
    [J]. PHYSICAL REVIEW LETTERS, 1993, 71 (23) : 3886 - 3889
  • [7] The Dynamics of Bimodular Continuous Attractor Neural Networks with Moving Stimuli
    Yan, Min
    Zhang, Wen-Hao
    Wang, He
    Wong, K. Y. Michael
    [J]. NEURAL INFORMATION PROCESSING (ICONIP 2017), PT IV, 2017, 10637 : 648 - 657
  • [8] Experimental Study of Multistability and Attractor Dynamics in Winnerless Neural Networks
    Chauhan, Ashok
    Nogaret, Alain
    [J]. ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2016, PT I, 2016, 9886 : 562 - 562
  • [9] Attractor Dynamics Driven by Interactivity in Boolean Recurrent Neural Networks
    Cabessa, Jeremie
    Villa, Alessandro E. P.
    [J]. ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2016, PT I, 2016, 9886 : 115 - 122
  • [10] Computational Capabilities of Recurrent Neural Networks Based on their Attractor Dynamics
    Cabessa, Jeremie
    Villa, Alessandro E. P.
    [J]. 2015 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2015,