Digital Signal Processing Reference
In-Depth Information
the following constrained gradient-search procedure
w
0
(
k þ
1)
¼ w
(
k
)
þmC
x
(
k
)
w
(
k
)
w
(
k þ
1)
¼ w
0
(
k þ
1)
=kw
0
(
k þ
1)
k
2
in which the stepsize
m
is sufficiency small enough. Using the approximation
m
2
m
yields
w
0
(
k þ
1)
=kw
0
(
k þ
1)
k
2
¼
[
I
n
þmC
x
(
k
)]
w
(
k
)
=
{
w
T
(
k
)[
I
n
þmC
x
(
k
)]
2
w
(
k
)}
1
=
2
[
I
n
þmC
x
(
k
)]
w
(
k
)
=
[1
þ
2
mw
T
(
k
)
C
x
(
k
)
w
(
k
)]
1
=
2
[
I
n
þmC
x
(
k
)]
w
(
k
)[1
mw
T
(
k
)
C
x
(
k
)
w
(
k
)]
w
(
k
)
þm
[
I
n
w
(
k
)
w
T
(
k
)]
C
x
(
k
)
w
(
k
)
:
Then, using the instantaneous estimate
x
(
k
)
x
T
(
k
)of
C
x
(
k
), Oja's neuron (4.23) is
derived.
Consider now the power method recalled in Subsection 4.2.4. Noticing that
C
x
and
I
n
þmC
x
have the same eigenvectors, the step
w
0
iþ
1
¼ C
x
w
i
of (4.9) can be replaced
by
w
0
iþ
1
¼
(
I
n
þmC
x
)
w
i
and using the previous approximations yields Oja's neuron
(4.23) anew.
Finally, consider the characterization of the eigenvector associated with the
unique largest eigenvalue of a covariance matrix derived from the mean square
error E
kxww
T
xk
recalled in Subsection 4.2.5. Because
2
¼
2
2
C
x
þC
x
ww
T
w
2
7
w
E
kxww
T
xk
þww
T
C
x
an unconstrained gradient-search procedure yields
w
(
k þ
1)
¼ w
(
k
)
m
[
2
C
x
(
k
)
þC
x
(
k
)
w
(
k
)
w
T
(
k
)
þw
(
k
)
w
T
(
k
)
C
x
(
k
)]
w
(
k
)
:
Then, using the instantaneous estimate
x
(
k
)
x
T
(
k
)of
C
x
(
k
) and the approximation
w
T
(
k
)
w
(
k
)
¼
1 justified by the convergence of the deterministic gradient-search pro-
cedure to
+u
1
when
m!
0, Oja's neuron (4.23) is derived again.
Furthermore, if we are interested in adaptively estimating the associated single
eigenvalue
l
1
, the minimization of the scalar function
J
(
l
)
¼
(
lu
1
C
x
u
1
)
2
by a
gradient-search procedure can be used. With the instantaneous estimate
x
(
k
)
x
T
(
k
)
of
C
x
(
k
) and with the estimate
w
(
k
)of
u
1
given by (4.23), the following stochastic
gradient algorithm is obtained.
l
(
k þ
1)
¼ l
(
k
)
þm
[
w
T
(
k
)
x
(
k
)
x
T
(
k
)
w
(
k
)
l
(
k
)]
:
(4
:
24)
We note that the previous two heuristic derivations could be extended to the adaptive
estimation of the eigenvector associated with the unique smallest eigenvalue of
C
x
(
k
). Using the constrained minimization (4.3) or (4.7) solved by a constrained
Search WWH ::
Custom Search