Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 112

Inhibitory Interactions imize the settling time associated with large constraint

satisfaction problems.

Bidirectional excitatory connectivity is like having a
microphone next to the speaker that it is driving (or
pointing a video camera at its own monitor output) —
you get feedback . To control runaway positive exci-
tation, the cortex has inhibitory neurons that activate
inhibitory synaptic channels on the excitatory neurons.
Two forms of inhibition are present in the cortex: feed-
forward (driven by the level of excitation coming into
a layer) and feedback (driven by the level of excitation
within the layer itself). The combination of these forms
of inhibition results in set point behavior, which oc-
curs around a point where more excitation leads to more
inhibition, resulting in less excitation, but less excita-
tion leads to less inhibition, resulting in more excitation.
Thus, the system has a preferred level of excitation. In-
stead of explicitly simulating all of the inhibitory neu-
rons, we can use a summary function that imposes set
point inhibition directly, resulting in greater efficiency
and simplicity. We use two forms of k winners take all
(kWTA) functions, where the set point is specified as
a parameter k neurons (out of n total neurons) that are
to be active at any given time. In addition to control-
ling excitation, inhibition results in a form of compe-
tition between neurons. This competition is a healthy
one, and it produces a selection pressure in activation
dynamics and learning that results in an evolution-like
survival (and adaptation) of the fittest representations.
Finally, it produces sparse distributed representations,
which make sense in terms of several aspects of the gen-
eral structure of our world.

3.8 Further Reading

Textbooks on neuropsychology or cognitive neuro-
science will provide a wealth of information about the
basic structure of the cortex and cognitive functions of
the different cortical areas. Bear, Conners, and Par-
adiso (1996) and Kalat (1995) are good for the ba-
sics, and Banich (1997) is good for cognitive functions
(as are the later chapters of this text). Also, Kandel
et al. (1991) is an often-used reference, though it is more
medically oriented.
White (1989a) provides a wealth of information
about the anatomical and physiological properties of the
cortex, and Shepherd (1990) contains a number of use-
ful chapters on the anatomy and physiology of a variety
of brain areas including the cortex and hippocampus.
Rakic (1994) provides an overview of the develop-
mental construction of the cortex.
Abbott and Sejnowski (1999) is a collection of papers
on detailed computational models on Neural codes and
distributed representations .
Hertz et al. (1991) provides a detailed introductory
treatment of constraint satisfaction, energy functions,
and the like.

Constraint Satisfaction

The combined effect of excitatory and inhibitory inter-
actions can be understood in terms of constraint sat-
isfaction , where the activity patterns in the network
evolve or settle over time in a way that maximizes the
satisfaction of constraints internal and external to the
network. This process can be understood mathemati-
cally by using an energy function , which summarizes
the overall level of satisfaction or harmony in the net-
work. We can show that the net effect of activation
propagation is always to increase the overall harmony
measure. Inhibition can improve the reliability and min-