State Initialization for Recurrent Neural Network Modeling of Time-Series Data

To use a Recurrent Neural Network (RNN) for time series modeling, it is essential to properly initialize the network, that is, to set the hidden neuron outputs properly at the initial time. Normally, an RNN is initialized with zero state values or at steady state. In the context of dynamic system identification, such initializations imply the system to be modelled is in steady state, i.e., capturing transient behaviour of the system is difficult if the network states are not properly initialized. If the network initial states are not calculable from the training data, then a method to infer them, both throughout the training and validation phases, is needed. In this paper, we use a feed forward neural network to initialize a structurally deep recurrent neural network in learning and multi-step prediction of the altitude of a real quadrotor vehicle. To the best of our knowledge, this is the first time a neural network has outperformed a physics based model for multi-step time series prediction from recorded quadrotor flight data.