Filtering-based Layer-wise Parameter Update Method for Training a Neural Network

Aiming at the difficulties in modeling variable nonlinear systems with noise interference, a network model with optimal generalization ability is established to realize the identification of the system. Traditional network parameter training method, like gradient descent method and least squares are centralized, and it is difficult to adaptively update the model parameters according to changes in the system. Firstly, in order to adaptively update the network parameters and quickly reflect the changes in the input and output of the system, the network weights are used as time-varying parameters, and some parameters in the network are updated by Kalman filtering algorithm. Then, in order to further improve the generalization ability of the network, EKF is used to update all the parameters in the network. Finally, the effectiveness of the algorithm is verified by an example of the standard data set UCI-ccpp.