Hierarchical Neural Networks for Multivariate Time Series Prediction

Considering the problem that for multivariate time series prediction, the adaptation of a single reservoir may not be sufficient to improve the prediction accuracy, we propose a novel hierarchical neural network herein. In the first hierarchy, several simplified echo state networks - simple cycle reservoirs (SCRs) are used to extract the dynamical features of the multivariate time series. Particle swarm optimization method is conducted in the pre-training stage to optimize the free parameters of SCRs. The reservoir states of SCRs are collected as dynamical features. In the second hierarchy, a feature selection method based on mutual information is used to select a compact feature set as the input for the extreme learning machine (ELM). In order to further improve the prediction accuracy, the optimal number of hidden nodes of the ELM is chosen by a modified recursive algorithm. Simulation results on monthly average temperature and rainfall series in Dalian China sustain that the proposed model is effective for multivariate time series.