Effective learning algorithm for restricted Boltzmann machines via spatial Monte Carlo integration

Restricted Boltzmann machines (RBMs) are a type of statistical machine learning model used in various applications. However, training RBM models is computationally difficult owing to the requirement of calculating expectations that have a combinatorial explosion. We provide a new and effective learning algorithm based on spatial Monte Carlo integration, which is an extension of the standard Monte Carlo integration and can approximate such intractable expectations with high accuracy. The proposed method exhibited superior performance compared to the standard learning method, i.e., contrastive divergence, in terms of accuracy and learning speed. However, there were cases in which the proposed learning method exhibited reduced performances. Thus, we further demonstrate that a heuristic initialization of the learning parameters can suppress this degradation.