Restricted Boltzmann machine: Recent advances and mean-field theory*

被引:39
|
作者
Decelle, Aurelien [1 ,2 ,3 ]
Furtlehner, Cyril [2 ,3 ]
机构
[1] Univ Complutense, Dept Fis Teor 1, Madrid 28040, Spain
[2] Univ Paris Saclay, INRIA Saclay, TAU Team, F-91405 Orsay, France
[3] Univ Paris Saclay, LISN, F-91405 Orsay, France
关键词
restricted Boltzmann machine (RBM); machine learning; statistical physics; STATISTICAL-MECHANICS; NEURAL-NETWORKS; MODEL; STORAGE;
D O I
10.1088/1674-1056/abd160
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
引用
收藏
页数:24
相关论文
共 50 条
  • [1] Restricted Boltzmann machine: Recent advances and mean-field theory
    Aurélien Decelle
    Cyril Furtlehner
    Chinese Physics B, 2021, (04) : 12 - 35
  • [2] Advanced mean-field theory of the restricted Boltzmann machine
    Huang, Haiping
    Toyoizumi, Taro
    PHYSICAL REVIEW E, 2015, 91 (05):
  • [3] Mean-Field Inference in Gaussian Restricted Boltzmann Machine
    Takahashi, Chako
    Yasuda, Muneki
    JOURNAL OF THE PHYSICAL SOCIETY OF JAPAN, 2016, 85 (03)
  • [4] Mean-field theory of Boltzmann machine learning
    Tanaka, T
    PHYSICAL REVIEW E, 1998, 58 (02) : 2302 - 2310
  • [5] A dynamical mean-field theory for learning in restricted Boltzmann machines
    Cakmak, Burak
    Opper, Manfred
    JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2020, 2020 (10):
  • [6] Variational mean-field theory for training restricted Boltzmann machines with binary synapses
    Huang, Haiping
    PHYSICAL REVIEW E, 2020, 102 (03)
  • [7] SPIN-GLASSES - RECENT ADVANCES IN MEAN-FIELD THEORY
    SOUTHERN, BW
    CANADIAN JOURNAL OF PHYSICS, 1987, 65 (10) : 1245 - 1250
  • [8] Boltzmann Machine and Mean-Field Approximation for Structured Sparse Decompositions
    Dremeau, Angelique
    Herzet, Cedric
    Daudet, Laurent
    IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2012, 60 (07) : 3425 - 3438
  • [9] Elastoplastic mean-field homogenization: recent advances review
    Sekkate, Zoubida
    Aboutajeddine, Ahmed
    Seddouki, Abbass
    MECHANICS OF ADVANCED MATERIALS AND STRUCTURES, 2022, 29 (03) : 449 - 474
  • [10] Theoretical study of mean-field Boltzmann machine learning by information geometry
    Arai, Toshiyuki
    Tanaka, Toshiyuki
    Fujimori, Yoritaka
    Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi), 1999, 82 (08): : 30 - 39