Probabilistic Forecasting of Symbol Sequences with Deep Neural Networks

Time series forecasting is usually done in a deterministic sense, such as in autoregressive moving average models, where a future state is predicted as a linear combination of past events. However, by formulating the problem in a probabilistic sense, soft predictions are obtained from a given probability mass function. This paper uses a deep neural network for probabilistic forecasting of time series by minimizing the cross entropy of the probability of future symbols from a given state. The advantage of this type of model is that it makes probabilistic inferences from the ground up, and without any restrictive assumptions (e.g., second order statistics). The efficacy of the proposed model is tested by forecasting the emergence of combustion instabilities, defined to be the root mean square of the pressure signal inside a laboratory-scale combustor system. The proposed algorithm has been compared with the autoregressive moving average (ARMA) model, which acts as a baseline for many time-series forecasting tasks, and the proposed model is shown to significantly outperform the ARMA model in this task.