Digital Signal Processing Reference
In-Depth Information
In the following, we will continue considering that the signals are real. The cor-
responding extensions for complex signals are straightforward and the reader can
obtain them easily.
4.2 NLMS Algorithm
It turns out that the LMS update at time
n
is a scaled version of the regression vector
x
, so the “size” of the update in the filter estimate is therefore proportional to
the norm of
x
(
n
)
. Such behavior can have an adverse effect on the performance of
LMS in some applications, e.g., when dealing with speech signals, where intervals
of speech activity are often accompanied by intervals of silence. Thus, the norm of
the regression vector can fluctuate appreciably. This issue can be solved by normal-
izing the update by
(
n
)
2
, leading to the
Normalized Least Mean Square
(NLMS)
algorithm. However, this algorithm might be derived in different ways, leading to
interesting interpretations on its operation mode.
x
(
n
)
4.2.1 Approximation to a Variable Step Size SD Algorithm
Consider the SD recursion (
4.1
) but now with a time varying step size
. The idea
is to find the step size sequence that achieves the maximum speed of convergence.
Particularly, at each time step the value
μ(
n
)
μ(
n
)
should be chosen to minimize the MSE
evaluated at
w
(
n
)
w
T
ξ(
n
)
= ˜
(
n
)
R
x
˜
w
(
n
).
(4.12)
Equation (
4.12
) is quadratic in
μ
, and its solution leads to the sequence
w
T
R
x
˜
˜
(
n
−
1
)
w
(
n
−
1
)
o
μ
(
n
)
=
)
.
(4.13)
w
T
R
x
˜
˜
(
n
−
1
)
w
(
n
−
1
Now, in order to get the stochastic gradient approximation algorithm, the true
statistics should be replaced using (
4.2
). The resulting update corresponds to the
NLMS, and has the form
x
(
n
)
w
(
n
)
=
w
(
n
−
1
)
+
2
e
(
n
).
(4.14)
x
(
n
)
So far, the NLMS can be viewed as a stochastic approximation to the SD with
maximum speed of convergence in the EMSE. Given the link between SD and LMS
algorithm, it might be possible to derive the NLMS from an LMS. Actually, (
4.14
)
