Restricted Boltzmann machine: Recent advances and mean-field theory*

被引:39
|
作者
Decelle, Aurelien [1 ,2 ,3 ]
Furtlehner, Cyril [2 ,3 ]
机构
[1] Univ Complutense, Dept Fis Teor 1, Madrid 28040, Spain
[2] Univ Paris Saclay, INRIA Saclay, TAU Team, F-91405 Orsay, France
[3] Univ Paris Saclay, LISN, F-91405 Orsay, France
关键词
restricted Boltzmann machine (RBM); machine learning; statistical physics; STATISTICAL-MECHANICS; NEURAL-NETWORKS; MODEL; STORAGE;
D O I
10.1088/1674-1056/abd160
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
引用
收藏
页数:24
相关论文
共 50 条
  • [31] MEAN-FIELD THEORY AND SOLITONIC MATTER
    COHEN, TD
    NUCLEAR PHYSICS A, 1989, 495 (3-4) : 545 - 563
  • [32] MEAN-FIELD THEORY OF THE PROTON GLASS
    DOBROSAVLJEVIC, V
    STRATT, RM
    PHYSICAL REVIEW B, 1987, 36 (16): : 8484 - 8496
  • [33] Boltzmann Machine learning using mean field theory and linear response correction
    Kappen, HJ
    Rodriguez, FB
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 10, 1998, 10 : 280 - 286
  • [34] MEAN-FIELD THEORY OF QUASICRYSTALLINE ORDER
    MERMIN, ND
    TROIAN, SM
    PHYSICAL REVIEW LETTERS, 1985, 54 (14) : 1524 - 1527
  • [35] MEAN-FIELD THEORY OF THE POTTS GLASS
    GROSS, DJ
    KANTER, I
    SOMPOLINSKY, H
    PHYSICAL REVIEW LETTERS, 1985, 55 (03) : 304 - 307
  • [36] MEAN-FIELD THEORY OF POLYMER MELTING
    BASCLE, J
    GAREL, T
    ORLAND, H
    JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1992, 25 (23): : L1323 - L1329
  • [37] Nonequilibrium dynamical mean-field theory
    Freericks, J. K.
    Turkowski, V. M.
    Zlatic, V.
    PHYSICAL REVIEW LETTERS, 2006, 97 (26)
  • [38] Mean-field theory of glass transitions
    Tokuyama, M
    PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS, 2006, 364 : 23 - 62
  • [39] Mean-field theory of inhomogeneous fluids
    Tschopp, S. M.
    Vuijk, H. D.
    Sharma, A.
    Brader, J. M.
    PHYSICAL REVIEW E, 2020, 102 (04)
  • [40] Mean-field theory of hot sandpiles
    Vergeles, M
    PHYSICAL REVIEW E, 1997, 55 (05): : 6264 - 6265