Research on Improved BP Algorithm by Computer Simulation Based on Adaptive Learning Rate

Starting from the problem that the structure and learning rules of the perceptron can't perform XOR, the BP network in the neural network is used to solve the XOR problem, which eliminates the limitations of the perceptron. However, there are often some problems in the implementation of the BP algorithm, such as slow convergence, strong coupling relationship with other parameters, local minima, etc. Therefore, based on the principle of feed forward neural network, an adaptive learning rate factor method is proposed to improve the BP algorithm, and the improved algorithm is used to learn two-dimensional XOR problems and multidimensional XOR problems. Simulation results show that the improved algorithm can significantly improve the learning speed of the network, and the learning process has good convergence and strong robustness. Because the standard BP algorithm is based on the steepest gradient descent method, it has some disadvantages, such as slow convergence speed, easy to fall into local minimum, difficult to determine the number of hidden layers and their nodes, and the expected output of each input mode must be known. Therefore, many scholars have done research and put forward some improved BP algorithms, including additional momentum method, adaptive learning rate method, elastic BP algorithm, Levenberg-Marquardt method, Gaussian-Newton method, conjugate gradient method, BP-GA algorithm and simulated annealing algorithm.