ANALYSIS OF THE MOMENTUM LMS ALGORITHM

Several modifications of the well-known LMS algorithm have been proposed for improved operation. This paper analyzes one such algorithm that corresponds to the standard LMS algorithm with an additional update term, parameterized by the scalar factor a where | a | < 1. The analysis of convergence yields some novel behavior insofar that it leads to complex eigenvalues of the transition matrix for the mean weight vector. It is demonstrated that the algorithm becomes unstable as \a\ → 1. Several computer simulation examples support the conclusion that, while the momentum LMS algorithm has smoother convergence, no significant gain in convergence speed over the conventional LMS algorithm can be expected. However, because of this smoothing effect, the MLMS algorithm may be useful in applications where error bursting is a problem. The results presented in this paper also illustrate some convergence properties of a nonlinear form of the MLMS algorithm, such as that used to train a single-layer perceptron. © 1990 IEEE