The generalized back-propagation algorithm with convergence analysis

The conventional back-propagation algorithm is basically a gradient-descent method, it has the problems of local minima and slow convergence. The generalized back-propagation algorithm which can effectively speed up the convergence rate has been proposed previously in [1]. In this paper, the convergence proof of the algorithm is being analyzed. The generalized back-propagation algorithm is to change the derivative of the activation function so as to magnify the backward propagated error signal when the output approaches a wrong value, thus the convergence rate can be accelerated and the local minimum can be escaped. From the convergence analysis, it is shown that the generalized back-propagation algorithm will improve the original backpropagation algorithm in terms of faster convergence and global search capability.