Mechanization of Cognition - Biomimetics: Biologically Inspired Technologies - page 105

Figure 3.A.3 Simple attractor network example. The left, x, neural field has N neurons; as does the right,
y, neural field. One Willshaw stable state pair, x k and y k is shown here (actually, each x k and y k typically has
many tens of neurons — e.g., Np ¼ 60 for the parameter set described in the text — of which only 10 are shown
here). Each neuron of each state sends connections to all of the neurons of the other (only the connections
from one neuron in x k and one neuron in y k are shown here). Together, the set of all such connections for all L
stable pairs is recorded in the connection matrix W. Notice that these connections are not knowledge links — they
are internal connections between x k and y k — the two parts of the neuron population of symbol k within
a single module. Also, unlike knowledge link connections (which, as discussed in the next section, are unidirec-
tional and for which the second stage is typically very sparse), these interpopulation connections must be
reciprocal and dense (although they need not be 100% dense — a fact that you can easily establish experimentally
with your model).
network was two reciprocally connected Willshaw networks; however, it also had an energy
function. Karen Haines and I theoretically investigated the dynamics of this network (Haines and
Hecht-Nielsen, 1988) [in 1988 computer exploration of the dynamics of such networks, at scales
sufficiently large to explore their utility for information processing, was not feasible]. We were able
to show theoretically that this hybrid had four important (and unique) characteristics. First, it
would, with very high probability, converge to one of the Willshaw stable states. Second, it would
converge in a finite number of steps. Third, there were no ''spurious'' stable states. Fourth, it could
carry out a ''winner take all'' kind of information processing. This hybrid network might thus serve
as the functional implementation of (in the parlance of this Appendix) a symbolic lexicon. This was
the first result on the trail to the theory presented here. It took another 16 years to discover that, by
having antecedent support knowledge links deliver excitation to symbols (i.e., stable states) of such
a lexicon, this simple one-winner-take-all information processing operation ( confabulation )is
sufficient to carry out all of cognition.
By 1992 it had become possible to carry out computer simulations of reciprocal Willshaw
networks of interesting size. This immediately led to the rather startling discovery that, even
without an energy function (i.e., carrying out neuron updating on a completely local basis, as in
Willshaw's original work), even significantly ''damaged'' (the parlance at that stage of discovery)
starting states (Willshaw stable states with a significant fraction of added and deleted neurons)
would almost always converge in one ''round-trip'' or ''out-and-back cycle.'' This made it likely
that this is the functional design of cortical lexicon circuits.
As this work progressed, it became clear that large networks of this type were even more robust
and would converge in one cycle even from a small incomplete fragment of a Willshaw stable state.
It was also at this point that the issue of ''threshold control'' (Willshaw's original neurons all had
the same fixed ''firing'' threshold — equal to the number of neurons in each stable state) came to the
fore. If such networks were operated by a threshold control signal that rose monotonically from a
minimum level, it could automatically carry out a global ''most excited neurons win'' competition
without need for communication between the neurons. The subset of neurons which become active
first then inhibit others from becoming so (at least in modules in the brain; but not in these simple
mathematical models, which typically lack inhibition). From this came the idea that each module
must be actively controlled by a graded command signal, much like an individual muscle. This