Information Technology Reference
In-Depth Information
dx
.t/ D Œ
Bx
.t/ C
Ag
.x.t// C C
dt
C
dw
.t/
(8.10)
In the above relation
w
.t/ D Œ
w
1
.t/; ;
w
m
.t/
T
is an m-dimensional Brownian
motion defined on a probability space .˝;F;P/with a natural filtration fF
t0
g, i.e.
F
t
D f
w
.s/ W 0stg and W R
n
! R
m
i.e. .x/ D .
ij
.x//
nm
which is called
the noise intensity matrix.
8.2
Interacting Diffusing Particles as a Model of Neural
Networks
8.2.1
Weights' Equivalence to Brownian Particles
An equivalent concept of a neural network with weights described by interacting
Brownian particles can be also found in [
83
,
87
]. Here, a neural structure of M
neurons is considered, e.g. an associative memory (Fig.
8.1
). For the weight
w
,the
error vector Œe; e is defined, where e denotes the distance of the weight from the
desirable value
w
and e the rate of change of e. Thus, each weight can be mapped
to the 2-D plane using the notation Œx;y D Œe; e. Moreover, each weight is taken to
correspond to a Brownian particle. The particles are considered to have mechanical
properties, that is to be subjected to acceleration due to forces. The objective of
learning is to lead a set of M weights (Brownian particles) with different initial
values on the 2-D plane, to the desirable final position Œ0;0.
Each particle (weight) is affected by the rest of the M 1 particles. The cost
function that describes the motion of the i-th particle towards the equilibrium is
denoted as V.x
i
/ W R
n
2
e
i
2
[
147
,
150
-
152
]. Convergence to
the goal state should be succeeded for each particle through the negative definiteness
of the associated Lyapunov function, i.e. it should hold
1
! R, with V.x
i
/ D
V
i
.x
i
/ De
i
.t/
T
e
i
.t/ < 0
[
92
].
As already mentioned, in the quantum harmonic oscillator (QHO) model of the
neural weights the update of the weight (motion of the particle) is affected by (1) the
drift force due to the harmonic potential, and (2) the interaction with neighboring
weights (particles). The interaction between the i-th and the j-th particle is [
64
]:
g.x
i
x
j
/ D.x
i
x
j
/Œg
a
.jjx
i
x
j
jj/ g
r
.jjx
i
x
j
jj/
(8.11)
where g
a
./ denotes the attraction term and is dominant for large values of jjx
i
x
j
jj,
while g
r
./ denotes the repulsion term and is dominant for small values of jjx
i
x
j
jj.
Function g
a
./ can be associated with an attraction potential, i.e.
r
x
i
V
a
.jjx
i
x
j
jj/ D .x
i
x
j
/g
a
.jjx
i
x
j
jj/
(8.12)
