A back-propagation algorithm with adaptive momentum factor

A method was developed to adaptively determine the momentum factor of back-propagation (BP) algorithm to enhance the training speed of the neural networks. Taking the learning rate as constant, the algorithm adjusted the momentum factor according to the gradient of the error function with respect to the weight vector. Numerical experiments show that the proposed algorithm is effective for both the batch and online training. Moreover, it is superior to the BP algorithm with constant momentum factor in respect of convergence rate and stability.