Information Technology Reference
In-Depth Information
dx
.t/
dt
D h.x.t// C .t/)
dx
.t/ D h.x.t//
dt
C
w
.t/
(7.66)
where h.x.t// D ˛
@V.x/
@t
, with ˛ being a learning gain, V.x/ being the harmonic
potential, and .t/ being a noise function. Equation (
7.64
) is a generalization of
gradient algorithms based on the ordinary differential equation (ODE) concept,
where the gradient algorithms are described as trajectories towards the equilibrium
of an ODE [
22
,
52
]. Indeed, conventional gradient algorithms with diminishing step
are written as
dx
.t/ D h.x.t//
dt
(7.67)
The comparison of Eqs. (
7.64
) and (
7.67
) verifies the previous argument. The update
of the neural weights that follow the model of the QHO given by Eq. (
7.64
). The
force that drives x.t/ to the equilibrium is the derivative of the harmonic potential,
and there is also an external noisy force
w
.t/ which is the result of collisions
or repulsive forces due to interaction with neighboring particles. Similarly, in the
update of neural weights with a conventional gradient algorithm, the weight is driven
towards an equilibrium under the effect of a potential's gradient (where the potential
is usually taken to be a quadratic error cost function).
7.7
Conclusions
In this chapter, neural structures with weights which are stochastic variables and
which follow the model of the QHO have been studied. The implications of
this assumptions for neural computation were analyzed. The neural weights were
taken to correspond to diffusing particles, which interact to each other as the
theory of Wiener process (Brownian motion) predicts. The values of the weights
are the positions of the particles and the probability density function
2
that describes their position is derived by Schrödinger's equation. Therefore the
dynamics of the neural network is given by the time-dependent solution of
Schrödinger's equation under a parabolic (harmonic) potential.
However, the time-dependent solution of Schrödinger's equation is difficult to
be computed, either analytically or numerically. Moreover, this solution contains
terms which are difficult to interpret, such as the complex number probability
amplitudes associated with the modes
k
.x/ of the wave-function .x/ (solu-
tion of Schrödinger's equation), or the path integrals that describe the particles'
motion. Thus in place of Schrödinger's equation the solution of a stationary
diffusion equation (Ornstein-Uhlenbeck diffusion) with drift was studied, assuming
probability density function that depends on the QHO's ground state
0
.x/ D
j
0
.x/j
j .x;t/j
2
.
