Recurrent Restricted Boltzmann Machine for Chaotic Time-series Prediction

How to extract effective information from large-scale time-series for prediction has become a hot topic in dynamic modeling. Benefiting from powerful unsupervised feature learning ability, restricted Boltzmann machine (RBM) has exhibited fabulous results in time-series feature extraction, and is more adaptive to input data than many traditional time-series prediction models. However, for the application of chaotic time-series prediction, RBM lacks a unique mechanism to capture time-series information. In addition, RBM only provides a feature extraction mechanism for dynamic modeling and cannot perform the task of regression prediction alone. In view of aforementioned problems, we propose a recurrent restricted Boltzmann machine (RRBM) to capture dynamic information and perform regression prediction by introducing a recurrent structure of leaky integral reservoir. This recurrent structure not only can remedy the dynamic characteristics, but also has a short-term historical information memory, which is more suitable for time-series applications. On this basis, a cross-layer connection is established between the feature layer of RBM and output layer of RRBM to achieve feature reuse and compensate for the missing information in the recurrent process. Experiments show that RRBM has smaller prediction error and higher information utilization.