Information Technology Reference
In-Depth Information
V.x/D
P
i
D
1
V
i
.x
i
/ C
2
P
i
D
1
P
j
D
1;j
¤
i
fV
a
.jjx
i
1
x
j
jj V
r
.jjx
i
x
j
jj/g)
V.x/D
P
i
D
1
V
i
.x
i
/ C
2
P
i
D
1
P
j
D
1;j
¤
i
fajjx
i
1
x
j
jj V
r
.jjx
i
x
j
jj/ and
r
x
i
V.x/D Œ
P
i
D
1
r
x
i
V
i
.x
i
/ C
2
P
i
D
1
P
j
D
1;j
¤
i
r
x
i
1
fajjx
i
x
j
jj V
r
.jjx
i
x
j
jj/g)
r
x
i
V.x/D Œ
P
i
D
1
r
x
i
V
i
.x
i
/ C
P
j
D
1;j
¤
i
.x
i
x
j
/fg
a
.jjx
i
x
j
jj/ g
r
.jjx
i
x
j
jj/g)
r
x
i
V.x/D Œ
P
i
D
1
r
x
i
V
i
.x
i
/ C
P
j
D
1;j
¤
i
.x
i
x
j
/fa g
r
.jjx
i
x
j
jj/g
and using Eq. (
8.19
) with
i
.t/ D 1 yields r
x
i
V.x/Dx
i
, and
X
iD1
r
x
i
V.x/
T
M
X
iD1
jj x
i
M
V.x/Dr
x
V.x/
T
) V.x/D
x
i
2
x D
jj
0
(8.30)
Therefore, in the case of a quadratic cost function it holds V.x/>0 and V.x/0
and the set C Dfx W V.x.t// V.x.0//g is compact and positively invariant.
Thus, by applying La Salle's theorem one can show the convergence of x.t/ to the
set M C; M Dfx W V.x/D 0g) M Dfx Wx D 0g.
8.3
Convergence of the Stochastic Weights to an Equilibrium
To visualize the convergence of the stochastic weights to an attractor, the interaction
between the weights (Brownian particles) can be given in the form of a distributed
gradient algorithm, as described in Eq. (
8.19
):
2
4
h.x
i
.t// C
i
.t/ C
3
5
;iD 1;2; ;M
X
M
x
i
.tC1/ D x
i
.t/C
i
.t/
g.x
i
x
j
/
j
D
1;j
¤
i
(8.31)
The term h.x.t/
i
/ Dr
x
i
V
i
.x
i
/ indicates a local gradient algorithm, i.e. motion in
the direction of decrease of the cost function V
i
.x
i
/ D
2
e
i
.t/
T
e
i
.t/.Theterm
i
.t/
is the algorithms step while the stochastic disturbance e
i
.t/ enables the algorithm to
escape from local minima. The term
P
jD1;j¤i
g.x
i
1
x
j
/ describes the interaction
between the i-th and the rest M 1 Brownian particles (feedback from neighboring
neurons as depicted in Figs.
10.1
and
7.5
).
In the conducted simulation experiments the multi-particle set consisted of ten
particles (weights) which were randomly initialized in the 2-D field Œx;y D Œe; e.
The relative values of the parameters a and b that appear in the potential of
Eq. (
8.14
) affected the trajectories of the individual particles. For a>bthe cohesion
