Digital Signal Processing Reference
In-Depth Information
3
2
1
0
-1
-2
-3
-4
-4
-2
0
2
4
6
w
0
FIGURE 3.10
Contours of the MSE cost function and directions of the eigenvectors of
R
.
The discussed example illustrates the efficiency of the steepest-descent
algorithm in searching for the optimal Wiener solution. However, it is worth
reconsidering the motivations in using such procedure, i.e., the idea of car-
rying out the processes of data acquisition and filter optimization jointly. In
fact, it is evident from (3.49) that the steepest-descent algorithm does not
accomplish such requirement, since the knowledge of statistical correlation
(matrix
R
and vector
p
) is as indispensable to calculate the gradient vector as
to solve the Wiener-Hopf equations. This means that an alternative method-
ology must be introduced in order to cross over the frontier from optimal
(fixed) to adaptive filtering definitively. To proceed with such discussion,
the fundamental idea of stochastic algorithm is now introduced.
3.4 The Least Mean Square Algorithm
The pioneer works on adaptive filtering date from the 1950s. From that
time, this field of research has been significantly developed and originated
a wide range of applications, methods, algorithms, and tools of analysis.
Moreover, adaptive filtering became a well-established discipline in modern
signal processing theory, with a number of relevant and classical textbooks,
like [32,100,139,194,262,304] to mention a few.
Among the great number of efficient techniques, the
LMS
algorithm is
classically considered to be the “most popular” one as well as the basis
of many others, the so-called LMS-based algorithms. The most usual and
accessible way to introduce the LMS algorithm is by means of the concept
of stochastic approximation. Such concept was first posed and theoretically
