Digital Signal Processing Reference
In-Depth Information
n
+
1
u
i
(
−
u
i
(
n
)
=
(
1
−
μλ
i
)
1
).
(3.11)
, which
goes along the direction defined by its associated eigenvector. In order for the algo-
rithm to converge as
n
Then, each eigenvalue
λ
i
determines the mode of convergence
(
1
−
μλ
i
)
, the misalignment vector (and its transformed version)
must vanish. Since (
3.11
) would be associated to an exponential behavior, the nec-
essary and sufficient condition for the stability of the SD algorithm would be
→∞
|
1
−
μλ
i
|
<
1
i
=
0
,
1
,...,
L
−
1
.
(3.12)
This shows that the stability of the algorithm depends only on
(a design parameter)
and
R
x
(or more precisely, its eigenvalues). To satisfy the stability condition, the step
size should be chosen according to
μ
2
λ
max
.
0
< μ <
(3.13)
Recalling the canonical form of
J
MSE
(
w
)
to write
L
−
1
(
n
+
)
u
i
(
−
2
1
J
MSE
(
n
)
=
J
MMSE
+
ξ(
n
)
=
J
MMSE
+
0
λ
i
(
1
−
μλ
i
)
1
).
(3.14)
i
=
The second term
is known as the
excess mean square error
(EMSE) and mea-
sures how far the algorithm is from theminimum. Equation (
3.14
) shows the evolution
through the error surface as a function of the iteration number, and is known as the
learning curve
or MSE curve. It is the result of the sum of
L
exponentials associated
to the natural modes of the algorithm. Since
ξ(
n
)
2
(
n
+
1
)
>
i
, when (
3.13
)
is satisfied the convergence is also monotonic (and EMSE goes to zero in steady
state). Clearly, the choice of
λ
i
(
1
−
μλ
i
)
0
∀
will not only affect the stability of the algorithm but
also its convergence performance (when stable). Actually, from the
L
modes of con-
vergence
μ
(
−
μλ
i
)
there will be one with the largest magnitude, that will give the
slowest rate of convergence to the associated component of the transformed vector
u
1
. Therefore, this will be the mode that determines the overall convergence speed
of the SD algorithm. It is then possible to look for a value of
(
n
)
that guarantees the
maximum overall rate of convergence by minimizing the magnitude of the slowest
mode, i.e.,
μ
μ
opt
=
argmin
μ
max
|
1
−
μλ
i
|
.
(3.15)
i
=
1
,...,
L
|
1
−
μλ
|
<
1
i
In looking for the
μ
opt
we only need to study the modes associated to
λ
max
and
λ
min
as a function of
μ
, since all the others will lie in between. For
μ < μ
opt
the mode
associated to
λ
max
has smaller magnitude than the one associated to
λ
min
.Asthe
reverse holds for
μ > μ
opt
, the optimal step size must satisfy the condition
