Deep Recurrent Support Vector Machine for Online Regression

This paper introduces a novel deep recurrent support vector regressor (DRSVR) model for online regression. DRSVR model is constructed by a state equation followed by an output construction. The inner layer is actually a least squares support vector regressor (LS-SVR) of the states with an adaptive kernel function. In addition, an infinite impulse response (IIR) filter is adopted in the model. LS-SVR and IIR filter together constitute an intermediate layer which performs the recursive state update. Each internal state has a recurrency which is a function of the observed input-output data and the previous states. Hence, internal states track the temporal dependencies in the feature space. The outer layer is a linear combination of the states. The model parameters, including the Gaussian kernel width parameter, are updated simultaneously, that provides the model to capture the time-varying dynamics of the data quickly. Parameters are adaptively tuned using error-square minimization via conventional Gauss-Newton optimization while keeping the poles of the IIR filter constrained to maintain stability. The proposed DRSVR model is applied for real-time nonlinear system identification. The identification results indicate the accurate regression performance of the proposed model.