Gradient adaptive digital filtering: Problems and solutions.

Abstract

The LMS adaptive algorithm has always been attractive to researchers in the field of adaptive signal processing due to its inherent conceptual and implementational simplicity. Unfortunately, this elegant simplicity is undermined by problems associated with the direct use of the LMS algorithm. One of the main disadvantages of the LMS is its relatively slow convergence. We deal with this problem for FIR adaptive filters by proposing two algorithms based on different approaches. The first algorithm relies on the time-varying step size approach. The step size of the algorithm is adjusted according to an error autocorrelation function. As a result, the algorithm can efficiently sense the adaptation state while maintaining the immunity against independent noise disturbance. The second algorithm is a gradient-based one that combines time- and order-updating when searching the bottom of the MSE surface, thus resulting in more efficient use of the available information. Moreover, two possibilities for the order update are considered: straightforward sequential or selective schemes. Approximate analysis of convergence and steady state performance of the two algorithms are provided. The slow convergence problem of the LMS algorithm is also investigated for IIR adaptive filters based on output-error formulation. A new adaptive algorithm is proposed. The algorithm combines the least mean square (LMS) method with its low complexity and the least squares method with its fast convergence into a coupled LMS-LS adaptive scheme. Simulation examples indicate that the proposed scheme converges significantly faster than the LMS with minimal increase in complexity. Next, we consider the Leaky LMS algorithm as an LMS variant proposed to deal with numerous problems that arise in direct application of LMS, including: lack of persistent excitation in the input sequence, stalling, bursting, etc. However, despite the wide spread usage of the Leaky LMS, there has been no detailed study of its performance. We present an analytical treatment of the mean square error for zero-mean Gaussian input data. Exact expressions for the second moment of the coefficient vector, the algorithm misadjustment, and rigorous conditions for MSE convergence are derived. Finally, we consider one of the common applications of the LMS algorithm, echo cancellation in telephone networks. We investigate the presence of bursting on a back-to-back hybrid connection. Based on the essential fact that the high cross correlation between the input to the adaptive echo canceler and the transmitted signal at the near-end is the root cause of the bursting problem, we modify the conventional echo canceler such that under bursting circumstances the cross-correlation is substantially reduced and bursting is averted. The proposed system ensures normal operation is not affected. Implementation details of the proposed system are studied.