Information Technology Reference
In-Depth Information
the theory of quantum mechanics and demonstrate the particle-wave nature of
information have been analyzed in [
34
,
120
,
127
,
133
]. The use of neural networks
compatible with quantum mechanics principles can be found in [
6
,
135
,
137
,
145
],
while the relation between random oscillations and diffusion equations has been
studied in [
66
,
179
]. Other studies on neural models with quantum mechanical
properties can be found in [
143
,
149
,
167
,
198
].
In this chapter it is assumed that the weights of neural networks are stochastic
variables which correspond to diffusing particles, and interact to each other as
the theory of Brownian motion (Wiener process) predicts. Brownian motion is the
analogous of the QHO, i.e. of Schrödinger's equation under harmonic (parabolic)
potential. However, the analytical or numerical solution of Schrödinger's equation
is computationally intensive, since for different values of the potential V.x/ it is
required to calculate the modes
k
.x/ in which the particle's wave-function .x/ is
decomposed. Moreover, the solution of Schrödinger's equation contains non-easily
interpretable terms such as the complex number probability amplitudes which are
associated with the modes
k
.x/, or the path integrals that constitute the particle's
trajectory. On the other hand, instead of trying to solve Schrödinger's equation for
various types of V.x/ one can study the time evolution of the particle through an
equivalent diffusion equation, assuming probability density function depending on
QHO's ground state, i.e.
0
.x/ Dj
0
.x/j
2
[
56
].
This chapter extends the results on neural structures compatible with quantum
mechanics principles presented in [
163
,
165
]. Moreover, the chapter extends the
results on quantum neural structures, where the basic assumption was that the neural
weights correspond to diffusing particles under Schrödinger's equation with zero
or constant potential [
149
]. The diffusing particle (stochastic weight) is subject to
the following forces: (1) a spring force (drift) which is the result of the harmonic
potential and tries to drive the particle to an equilibrium and (2) a random force
(noise) which is the result of the interaction with neighboring particles. This
interaction can be in the form of collisions or repulsive forces. It is shown that
the diffusive motion of the stochastic particles (weights' update) can be described
by Langevin's equation which is a stochastic linear differential equation [
56
,
66
].
Following the analysis of [
22
,
52
,
93
] it is proven that Langevin's equation is a
generalization of the conventional gradient algorithms. Therefore neural structures
with crisp numerical weights can be considered as a subset of NN with stochastic
weights that follow the QHO model.
7.2
Wiener Process and Its Equivalence to Diffusion
7.2.1
Neurons' Dynamics Under Noise
Up to now, neuronal dynamics has been expressed in a deterministic manner.
However, stochastic terms such as noise may also be present in the neurons'
