Information Technology Reference
In-Depth Information
The term
A
in eq. (3.17) is the anti-Hebbian term. The term
B
is the con-
straining term (forgetting term). The term
C
performs the Gram-Schmidt
orthonormalization. Oja [141] demonstrated that under the assumptions
w
=
0,
w
j
(
0
)
2
=
1,
T
j
λ
N
−
M
+
1
<
1, and
ϑ > λ
N
−
M
+
1
/λ
N
−
1,
(
0
)
z
j
the weight vector
in eq.
(3.18) converges to the minor components
...
ϑ
λ
N
−
M
+
1
/λ
N
−
z
N
−
M
+
1
,
z
N
−
M
+
2
,
,
z
N
.If
is larger than but close to
1,
the convergence is slow.
•
LUO MSA algorithms
. In [121],
two learning laws are proposed. The
first is
T
j
(
t
) w
j
(
t
)
x
(
t
)
+
α (
t
)
y
j
w
j
(
t
+
1
)
=
w
j
(
t
)
−
α (
t
)
y
j
(
t
) w
(
t
) w
j
(
t
)
(
t
) w
j
(
t
)
i
T
j
+
βα (
t
)
y
j
(
t
) w
y
i
(
t
) w
i
(
t
)
(3.19)
>
j
=
N
,
N
−
...
,
N
−
M
+
for
j
1,
1. The corresponding averaging ODE is
given by
d
w
j
(
t
)
dt
I
T
j
T
i
=−
w
(
t
) w
j
(
t
)
−
β
j
w
i
(
t
) w
(
t
)
i
>
T
j
R
w
(
t
)
+
w
(
t
)
R
w
(
t
) w
(
t
)
(3.20)
j
j
j
where
β
is a positive constant affecting the convergence speed. Equation
(3.20) does not possess the invariant norm property (2.27) for
j
<
N
.The
second (MSA LUO) is the generalization of the LUO learning law (2.25):
w
j
(
t
+
1
)
=
w
j
(
t
)
−
α (
t
)
y
j
(
t
)
(3.21)
T
j
T
j
w
(
t
) w
(
t
)
x
j
(
t
)
−
w
(
t
)
x
j
(
t
) w
(
t
)
j
j
for
j
=
N
,
N
−
1,
...
,
N
−
M
+
1and
x
j
(
t
)
=
x
(
t
)
−
i
y
i
(
t
) w
i
(
t
)
(3.22)
>
j
for
j
=
N
−
1,
...
,
N
−
M
+
1. Equation (3.22)
represents,
at
con-
vergence,
the projection of
x
(
t
)
on the subspace orthogonal
to the
...
eigenvectors
z
j
+
1
,
z
j
+
2
,
,
z
N
. In this way, an adaptive Gram-Schmidt
orthonormalization is done. Indeed, note that if
T
T
T
w
(
0
)
z
N
=
w
(
0
)
z
N
=···=
w
(
0
)
z
i
=
0
(3.23)
in the assumptions for Theorem 49, the LUO weight vector converges to
z
i
−
1
(this is clear in the proof of Theorem 60). This idea has been proposed
Search WWH ::
Custom Search