Information Technology Reference
In-Depth Information
Theorem 60 (Asymptotic Behavior)
Let R be the n
×
n autocorrelation
matrix input data, with eigenvalues
0
≤
λ
n
≤
λ
n
−
1
≤···≤
λ
1
and correspond-
ing orthonormal eigenvectors z
n
,
z
n
−
1
,
...
,
z
1
.If
w (
0
)
T
(
0
)
z
n
=
0
and
λ
n
is single, then,
in the limits of validity of the ODE approximation
,for
MCA EXIN it holds that
satisfies
w
w (
t
)
→±
w (
0
)
2
z
n
∨
w (
t
)
→∞
(2.79)
2
2
2
2
w (
t
)
≈
w (
0
)
t
>
0
(2.80)
T
w
Proof.
(also deals with OJAn and LUO) Multiplying eq. (2.33) by
(
t
)
on
the left yields
2
2
−
w (
)
+
w
)
w (
d
w (
t
)
2
w
(
t
)
2
T
2
=
t
)
2
R
w (
t
(
t
)
R
w (
t
t
)
4
2
dt
=
0
(2.81)
Then
2
2
2
2
t
>
0
w (
t
)
=
w (
0
)
(2.82)
The symbol
≈
in eq. (2.80) is a consequence of the fact that the ODE represents
the learning law only as a first approximation. The same reasoning also applies to
OJAn and LUO. As demonstrated in Section 2.4, OJAn, LUO, and MCA EXIN
are gradient flows with different Riemannian metrics of the Rayleigh quotient.
It is well known (see, e.g., [84,118]) that the gradient flows always converge
to a minimum straight line. Indeed, for all three neurons, the Rayleigh quotient
can be chosen as a Lyapunov function for their ODEs. Restricting the RQ to a
sphere with the origin at the center justifies its use as a Lyapunov function
V
(w)
because it is definite positive and, unless a constant,
d
w (
t
)
dt
T
dV
(w)
d
w
dV
(w (
t
))
=
dt
n
−{
0
}
d
w (
t
)
dt
T
w (
t
)
−
2
2
=
(
∇
V
(w))
S
n
−
1
d
T
w (
t
)
w
(
t
)
−
2
Q
(
w
(
t
))
=
grad
V
(
w
)
dt
d
w (
t
)
dt
T
d
w (
t
)
dt
w (
t
)
−
2
Q
(w (
t
))
=−
2
d
w (
t
)
dt
=−
Q
(w (
t
))
w (
t
)
−
2
2
2
Search WWH ::
Custom Search