Biomedical Engineering Reference
In-Depth Information
Spurious patterns are stable states, that is, local
minima of the corresponding energy function of
the network, not associated to any stored (input)
pattern. The simplest, but not the least important,
case of apparition of spurious patterns is the fact
of storing, given a pattern, its opposite, i.e. both
X
and
-X
are stable states for the net, but only one of
them has been introduced as an input pattern.
The problem of spurious patterns is very
fundamental for cognitive modelers as well as
practical users of neural networks. Many solutions
have been suggested in the literature. Some of
them (Parisi, 1986) (Hertz et al., 1987) are based
on introducing asymmetry in synaptic connec-
tions. However, it has been demonstrated that
synaptic asymmetry does not provide by itself a
satisfactory solution to the problem of spurious
patterns, see (Treves et al., 1988) (Singh et al.,
1995). Athitan et al. (1997) provided a solution
based on neural self-interactions with a suitably
chosen magnitude, if Hebb's learning rule is used,
leading to the near (but not) total suppression of
spurious patterns.
Crick (1983) suggested the idea of unlearning
the spurious patterns as a biologically plausible
solution to suppress them. With a physiological
explanation, they suggest that spurious patterns
are unlearned randomly by human brain during
sleep, by means of a process that is the reverse
of Hebb's learning rule. This may result in the
suppression of many spurious patterns with large
basins of attraction. Experiments have shown that
their idea leads to an enlargening of the basins
for correct patterns along with the elimination of
a significant fraction of spurious patterns (van
Hemmen et al., 1991). However, a great number of
spurious patterns with small basins of attraction
do survive. Also, in the process of undiscriminate
reverse learning, there is a finite probability of
unlearning correct patterns, what makes this
strategy unacceptable.
On the other hand, the capacity parameter α
is usually defined as the quotient between the
maximum number of patterns to load into the
network, and the number of used neurons that
achieve an acceptable error probability in the
retrieving phase, usually
p
e
=
0.01 or
p
e
=
0.05.
It was empirically shown that this constant is
approximately
= 0.15
for BH (very close to its
actual value,
= 0.1847
, see (Hertz et al., 1991)).
The meaning of this capacity parameter is that,
if the net is formed by
N
neurons, a maximum
of
K
≤
patterns can be stored and retrieved
with little error probability.
McElliece et al. (1987) showed that an upper
bound for the asymptotic capacity of the network
is
1
2 log
N
, if most of the input (prototype) patterns
are to remain as fixed points. This capacity de-
creases to
1
4 log
N
if every pattern must be a fixed
point of the net.
By using Markov chains to study capacity
and the recall error probability, Ho et al. (1992)
showed results very similar to those obtained by
McEliece, since for them it is
= 0.12
for small
values of
N
, and the asymptotical capacity is
given by
1
4 log
N
.
Kuh et al. (1989) manifested roughly similar
estimations by making use of normal approxima-
tion theory and the theorems about exchangeables
random variables.
Hopfield's Model
Hopfield's bipolar model consists in a network
formed by
N
neurons, whose outputs (states) belong
to the set {-1,1}. Thus, the state of the net at time
t
is completely defined by a
N
-dimensional state
vector
{ 1,1}
N
V
.
Associated to every state vector there is an
energy function, expressed in the following
terms:
( ) = (
t
V t
( ),
V
( ),
t
,
V
( ))
t
∈ −
1
2
N
1
N
N
N
∑∑
∑
E
(
V
) =
−
w V V
+
V
(1.1)
i j
,
i
j
i
i
2
i
=1
j
=1
i
=1
where
w
i,j
is the connection weight between neu-
rons
i
and
j
, and
i
is the threshold corresponding
to
i
i-th neuron (since thresholds are not used in the
case of associative memory, from now on all of
Search WWH ::
Custom Search