The stability of variable step-size LMS algorithms

Variable step-size LMS (VSLMS) algorithms are a popular approach to adaptive filtering, which can provide improved performance while maintaining the simplicity and robustness of conventional fixed step-size LMS. Here, we examine the stability of VSLMS with uncorrelated stationary Gaussian data. Most VSLMS described in the literature use a data dependent step-size, where the step-size either depends on the data before the current time (prior step-size rule) or through the current time (posterior step-size rule). It has often been assumed that VSLMS algorithms are stable tin the sense of mean-square bounded weights), provided that the step-size is constrained to lie within the corresponding stability region for the LMS algorithm. For a single tap filter, we find exact expressions for the stability region of VSLMS over the classes of prior and posterior step-sizes and show that the stability region for prior step-size coincides with that of fixed step-size, but the region for posterior step-size is strictly smaller than for fixed step-size. For the multiple tap case, me obtain bounds on the stability regions with similar properties. The approach taken here is a generalization of the classical method of analyzing the exponential stability of the weight covariance equation for LMS, Although it is not possible to derive a weight covariance equation for general data-dependent VSLMS, the weight variances can be upper bounded by the solution of a linear time-invariant difference equation, after appropriately dealing with certain nonlinear terms. For prior step-size (like fixed step-size), the state matrix is symmetric, whereas for posterior step-size, the symmetry is lost, requiring a more detailed analysis. The results are verified by computer simulations.