A New Levenberg-Marquardt Algorithm for feedforward neural networks

In this paper, a new fast training algorithm for feedforward neural networks is presented. This algorithm, named New Levenberg-Marquardt (NLM) Algorithm, is proposed based on the idea of the famous Levenberg-Marquardt (LM) training algorithm. The feedforward neural networks have a very good property that when one of the parameters e.g. a weight or a threshold changes, only the variables in relation with it will be affected, while others remain fixed. According to this characteristic, we can optimize the parameters one by one with the corresponding simplified error functions. Four classes of solution equations for parameters of the networks are deducted respectively. In this way, the computation complexity is decreased to a certain degree. After that, the famous Levenberg-Marquardt method is used to solve the optimization problem. Effectiveness of the presented algorithm is demonstrated by two benchmarks, in which faster convergence rate of training are obtained in contrast with the BP algorithm with momentum (BPM), the conjugate gradient (CG) algorithm, and the pure LM algorithm.