Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 91

range of initial states that lead to the same final attrac-
tor state comprise the attractor basin for the attractor
(think of a wash basin or sink).
In part for the reasons we just encountered in the pre-
vious exploration, a fuller treatment and exploration of
attractors awaits the incorporation of inhibitory dynam-
ics and the more general constraint satisfaction frame-
work described later in section 3.6. Nevertheless, it is
useful to redescribe some of the effects of bidirectional
excitatory connectivity that we just explored in terms of
attractor dynamics.
For example, one can think of pattern completion as
the process of the network being attracted toward the
stable state of the complete pattern from any of a num-
ber of different partial initial states. Thus, the set of
partial initial states that lead to the complete state con-
stitute the basin of attraction for this attractor.
Similarly, the bootstrapping effects of bidirectional
amplification can be seen as reflecting a starting point
out in the shallow slope of the farthest reaches of the
attractor basin. As the activation strength slowly builds,
the network gets more and more strongly pulled in to-
ward the attractor, and the rate of activation change in-
creases dramatically. Furthermore, the problematic case
where the activation spreads to all units in the network
can be viewed as a kind of superattractor that virtually
any initial starting state is drawn into.
In section 3.6, we will see that the notion of an at-
tractor actually has a solid mathematical basis not so far
removed from the metaphor of a gravity well depicted
in figure 3.18. Further, we will see that this mathemat-
ical basis depends critically on having symmetric bidi-
rectional excitatory connections, along with inhibition
to prevent the network from being overcome by a single
superattractor.

viewed as representing the likelihoods of two different
kinds of null hypotheses in the Bayesian hypothesis test-
ing framework analysis of detector function, see sec-
tion 2.7). In this section, we will explore the unique
contribution of the inhibitory counterweight.
As we saw in previous explorations, an important
limitation of the leak current as a counterweight is that
it is a constant . Thus, it cannot easily respond to dy-
namic changes in activation within the network (which
is why you have had to manipulate it so frequently).
In contrast, inhibition can play the role of a dynamic
counterweight to excitatory input, as a function of the
inhibitory input provided by the inhibitory interneurons
known to exist in the cortex (section 3.2). These neu-
rons appear to sample the general level of activation in
the network, and send a dynamically adjusted amount
of inhibition based on this activation level to nearby ex-
citatory neurons.
The role of the inhibitory interneurons can be viewed
like that of a thermostat-controlled air conditioner that
prevents the network from getting too “hot” (active).
Just as a thermostat samples the temperature in the air,
the inhibitory neurons sample the activity of the net-
work. When these inhibitory neurons detect that the
network is getting too active, they produce more inhi-
bition to counterbalance the increased activity, just as a
thermostat will turn on the AC when the room gets too
hot. Conversely, when they don't detect much activity,
inhibitory neurons don't provide as much inhibition.
Thermostats typically have a set point behavior,
where they maintain a roughly constant indoor tempera-
ture despite varying levels of heat influx from the exter-
nal environment. The set point is achieved through the
kind of negative feedback mechanism just described
— the amount of AC output is proportional to the ex-
cess level of heat above the set point.
In the cortex there are two forms of connectivity in-
volving the inhibitory neurons and their connections
with the principal excitatory neurons, which give rise
to feedforward and feedback inhibition (figure 3.19).
As we will see in the following explorations, both types
of inhibition are necessary. Note also that the inhibitory
neurons inhibit themselves, providing a negative feed-
back loop to control their own activity levels, which will
turn out to be important.

3.5 Inhibitory Interactions

In virtually every simulation encountered so far, the leak
current has played a central role in determining the net-
work's behavior, because the leak current has been the
only counterweight to the excitatory input coming from
other neurons. However, as we discussed in the previ-
ous chapter (section 2.4.6), the neuron actually has two
such counterweights, leak and inhibition (which can be