Restricted Boltzmann machine: Recent advances and mean-field theory*

被引:39
|
作者
Decelle, Aurelien [1 ,2 ,3 ]
Furtlehner, Cyril [2 ,3 ]
机构
[1] Univ Complutense, Dept Fis Teor 1, Madrid 28040, Spain
[2] Univ Paris Saclay, INRIA Saclay, TAU Team, F-91405 Orsay, France
[3] Univ Paris Saclay, LISN, F-91405 Orsay, France
关键词
restricted Boltzmann machine (RBM); machine learning; statistical physics; STATISTICAL-MECHANICS; NEURAL-NETWORKS; MODEL; STORAGE;
D O I
10.1088/1674-1056/abd160
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
引用
收藏
页数:24
相关论文
共 50 条
  • [41] Dynamics of polymers: A mean-field theory
    Fredrickson, Glenn H.
    Orland, Henri
    JOURNAL OF CHEMICAL PHYSICS, 2014, 140 (08):
  • [42] Mean-field theory for inhomogeneous electrolytes
    Yeh, SS
    Chen, P
    PHYSICAL REVIEW E, 2005, 72 (03):
  • [43] Mean-field theory for car accidents
    Huang, DW
    Tseng, WC
    PHYSICAL REVIEW E, 2001, 64 (05): : 4
  • [44] Dynamical mean-field theory for perovskites
    Lombardo, P
    Avignon, M
    Schmalian, J
    Bennemann, KH
    PHYSICAL REVIEW B, 1996, 54 (08): : 5317 - 5325
  • [45] Mean-field theory of hot sandpiles
    Phys Rev E., 5-B pt B (6264):
  • [46] MEAN-FIELD THEORY OF EQUILIBRIUM POLYMERIZATION
    JARIC, MV
    PHYSICAL REVIEW A, 1983, 27 (06): : 3342 - 3344
  • [47] Mean-field theory of the Potts gas
    Georgii, Hans-Otto
    Miracle-Sole, Salvador
    Ruiz, Jean
    Zagrebnov, Valentin A.
    JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 2006, 39 (29): : 9045 - 9053
  • [48] Mean-field theory of Bayesian clustering
    Mozeika, Alexander
    Coolen, Anthony C. C.
    PHYSICAL REVIEW E, 2018, 98 (04)
  • [49] Dynamical mean-field theory for bosons
    Anders, Peter
    Gull, Emanuel
    Pollet, Lode
    Troyer, Matthias
    Werner, Philipp
    NEW JOURNAL OF PHYSICS, 2011, 13
  • [50] ANTISYMMETRIZATION OF A MEAN-FIELD THEORY OF COLLISIONS
    GIRAUD, BG
    NAGARAJAN, MA
    NOBLE, CJ
    PHYSICAL REVIEW A, 1986, 34 (02): : 1034 - 1042