Digital Signal Processing Reference
In-Depth Information
algorithm, reads
C
(
k þ
1)
¼ C
(
k
)
þam
k
[
x
(
k
)
x
T
(
k
)
C
(
k
)]
(4
:
84)
W
(
k þ
1)
¼W
(
k
)
þm
k
[
I
n
W
(
k
)
W
T
(
k
)]
C
(
k
)
W
(
k
)
(4
:
85)
where
a
is introduced in order to normalize both algorithms because if the learning
rate of (4.84) has no dimension, the learning rate of (4.85) must have the dimension
of the inverse of the power of
x
(
k
). Furthermore
a
can take into account a better trade-
off between the misadjustments and the learning speed. Note that the performance
derivations may be extended to this smoothed Oja's algorithm by considering that
the coupled stochastic approximation algorithms (4.84) and (4.85) can be globally
written as (4.68) as well. Reusing now the parametrization (
u
ij
)
1
ijn
because
C
(
k
)
is symmetric as well, and following the same approach, we obtain now [25]
X
a
ij
l
i
l
j
2(
l
i
l
j
)
(
u
i
u
j
þu
j
u
i
)(
u
i
u
j
þu
j
u
i
)
T
C
P
¼
(4
:
86)
1
ir
,
jn
def
a=
(
aþl
i
l
j
)
,
1.
This methodology has been applied to compare the theoretical asymptotic perform-
ance of several minor and principal subspace adaptive algorithms in [24, 25]. For
example, the asymptotic mean square error E(
kP
(
k
)
P
k
with
a
ij
¼
2
Fro
) of the estimate
P
(
k
)
given by Oja's algorithm (4.30) and the smoothed Oja's algorithm (4.84) are shown
in Figure 4.2, where the stepsize
m
of the Oja's algorithm and the couple (
m
,
a
)of
the smoothed Oja's algorithm are chosen to provide the same value for
m
Tr(
C
P
).
We clearly see in this figure that the smoothed Oja's algorithm with
a¼
0.3 provides
faster convergence than the Oja's algorithm.
Regarding the issue of asymptotic bias, note that there is a real methodological pro-
blem to apply the methodology of the end of Subsection 4.7.3. The trouble stems from
the fact that the matrix
P
(
k
)
¼ W
(
k
)
W
T
(
k
) does not belong to a linear vector space
because it is constrained to have fixed rank
r
,
n
. The set of such matrices is not invar-
iant under addition; it is actually a smooth submanifold of
R
nn
. This is not a problem
in the first-order asymptotic analysis because this approach amounts to approximating
this manifold by its tangent plane at a point of interest. This tangent plane is linear
indeed. In order to refine the analysis by developing a higher order theory, it becomes
necessary to take into account the curvature of the manifold. This is a tricky business.
As an example of these difficulties, one could show (under simple assumptions) that
there exists no projection-valued estimators of a projection matrix that are unbiased at
order
O
(
m
); this can be geometrically pictured by representing the estimates as points
on a curved manifold (here: the manifold of projection matrices).
Using a more involved expression of the covariance of the field (4.74), the pre-
viously described analysis can be extended to correlated data
x
(
k
). Expressions
(4.83) and (4.86) extend provided that
l
i
l
j
is replaced by
l
i
l
j
þl
i,j
where
l
i,j
is
Search WWH ::
Custom Search