Information Technology Reference
In-Depth Information
(
I
is the identity matrix), which is the average version of the continuous-time
differential equation
x
(
t
)
x
T
T
(
t
)
x
(
t
)
x
T
d
w (
t
)
dt
(
t
)
−
w
(
t
) w (
t
)
1
w (
t
)
=−
w (
t
)
(2.34)
2
2
2
2
w (
t
)
which, after discretization, gives the MCA EXIN nonlinear stochastic learning
rule:
x
)
−
α (
t
)
y
(
t
)
y
(
t
) w (
t
)
w (
t
+
1
)
=
w (
t
(
t
)
−
(2.35)
2
2
2
2
w (
t
)
w (
t
)
where the instantaneous cost function is given by
T
xx
T
y
2
w
=
w
w
J
=
(2.36)
T
2
2
w
w
According to the stochastic approximation theory (see [113,118,142]), if some
conditions are satisfied, eq. (2.33) represents eq. (2.35) effectively (i.e., their
asymptotic paths are close with a large probability) and eventually the MCA
EXIN solution tends with probability 1 to the uniformly asymptotically stable
solution of the ODE. From a computational point of view, the most important
conditions are the following:
1.
x
(
t
)
is zero mean, stationary, and bounded with probability 1.
2.
is a decreasing sequence of
positive
scalars.
3.
t
α (
α (
t
)
t
)
=∞
.
4.
t
α
p
(
t
) <
∞
for some
p
.
5. lim
t
→∞
sup
1
1
α (
t
−
1
)
α (
t
)
−
<
∞
.
For example, the sequence
α (
t
)
=
const
·
t
−
γ
satisfies the four last conditions
for 0
<γ
≤
1. The fourth condition is less restrictive than the Robbins-Monro
condition [161]
t
α
2
(
t
) <
∞
, which is satisfied, for example, only by
α (
t
)
=
·
t
−
γ
with
2
<γ
≤
const
1.
MCA EXIN shares the same structure of LUO and OJAn:
w (
t
+
1
)
=
w (
t
)
−
α (
t
)
S
neuron
(w (
t
)
,
x
(
t
))
(2.37)
where
1
w (
1
w (
S
MCA EXIN
=
S
OJAn
=
S
LUO
(2.38)
2
2
4
2
t
)
t
)
The difference among these laws seems irrelevant, above all considering that
according to eq. (2.27), the weight modulus is constant. Instead, they behave very
differently. The reason for this is explained in the following section in Proposition
52 and in Theorem 68.
Search WWH ::
Custom Search