Spectral Analysis of Neural Models with Stochastic Weights - Advanced Models of Neural Networks - page 216

11.3.3 Stochastic weights and the Principle of Uncertainty

The significance of Eq. ( 11.39 ) is that the product between the information in the
space domain g. w ij / and the information in the frequency domain G.s/ cannot be
smaller than a constant. In the case of a Gaussian function g. w ij / D e

w ij 2
2 2

with

1
2 s 2 2

Fourier transform G.s/ D e

it can be observed that: (a) if is small, then
g. w ij / hasapickat w ij D 0 while G.s/ tends to become flat, (b) if is large, then
g. w ij / is flat at w ij D 0 while G.s/ makes a peak at s D 0.
This becomes more clear if the dispersion of function g. w ij / D e

w 2
ij
2 2 round
w ij D 0 is used [ 139 ]. The dispersion of g. w ij / and of its Fourier transform G.s/
becomes

w 2
ij
2 d w ij

R C1
1

2

w ij e

1

D.g/ D D

2 2

e 2 w ij

R C1
1

2 d w ij

(11.40)

R C1
1

2 s 2 2

s 2 e

ds

1
2 2

D.G/ D D

R C1
1

1
2 s 2 2

e ds

which results into the uncertainty principle for the weights of quantum associative
memories

1
4 :

D.g/D.G/ D (11.41)

Equation ( 11.41 ) means that the accuracy in the calculation of the weight w ij
is associated with the accuracy in the calculation of its spectral content. When
the spread of the Gaussians of the stochastic weights is large (small) then their
spectral content is poor (rich). Equation ( 11.41 ) is an analogous of the quantum
mechanics uncertainty principle, i.e. xp „, where x is the uncertainty in the
measurement of particle's position, p is the uncertainty in the measurement of the
particle's momentum, and „ is Planck's constant.
It should be noted that ( 11.41 ) expresses a general property of the Fourier
transform and that similar relations can be found in classical physics. For instance,
in electromagnetism it is known that it is not possible to measure with arbitrary
precision, at the same time instant, the variation of a wave function both in the
time and frequency domain. What is really quantum in the previous analysis is the
association of a wave function with a particle (stochastic weight) and the assumption
that the wave length and the momentum of the particle satisfy the relation p D„k
with jkjD

2

.