Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 111

an important new language for describing the dynam-
ics of cognition — including transforming patterns to
emphasize and deemphasize distinctions, settling into
an attractor, bootstrapping, inhibitory competition, and
constraint satisfaction. We will see in the next chap-
ter that by combining this language with principles of
learning, we have a very powerful toolkit for both un-
derstanding why networks of neurons (and by extension
the brains of animals and people) behave in the ways
they do, and for getting them to behave in the ways we
want them to.

feedforward connectivity pattern is where one set of
neurons connects with another set, but not vice versa.
This is not typically found in the cortex, but many prop-
erties of feedforward processing generalize to the more
common bidirectional case. Unidirectional excitatory
connectivity can transform input activity patterns in a
way that emphasizes some distinctions, while at the
same time deemphasizing or collapsing across other
possible distinctions. Much of cognition can be un-
derstood in terms of this process of developing repre-
sentations that emphasize relevant distinctions and col-
lapse across irrelevant ones. Distributed representa-
tions, where each unit participates in the representation
of multiple inputs, and each input is represented by mul-
tiple units, have a number of desirable properties, mak-
ing them predominant in our models and apparently in
the cortex. Localist representations, where a single unit
represents a given input, are less powerful, but can be
useful for demonstration purposes.
Bidirectional (a.k.a. recurrent or interactive ) con-
nectivity is predominant in the cortex, and has several
important functional properties not found in simple uni-
directional connectivity. We emphasize the symmetric
case (i.e., where both directions have the same weight
value), which is relatively simple to understand com-
pared to the asymmetric case. First, it is capable of per-
forming unidirectional-like transformations, but in both
directions, which enables top-down processing simi-
lar to mental imagery . It can also propagate informa-
tion laterally among units within a layer, which leads
to pattern completion when a partial input pattern is
presented to the network and the excitatory connections
activate the missing pieces of the pattern. Bidirectional
activation propagation typically leads to the amplifica-
tion of activity patterns over time due to mutual excita-
tion between neurons. There are several other important
subtypes of amplifying effects due to bidirectional ex-
citatory connections, including: mutual support , top-
down support or biasing, and bootstrapping .Manyof
these phenomena are described under the general term
of attractor dynamics , because the network appears to
be attracted to a particular activation state.

General Structure of Cortical Networks

The cortex is typically described as having six differ-
ent layers of neurons, but these can be categorized into
three primary groups or functional layers (figure 3.3),
the: input (cortical layer 4), hidden (cortical layers
2 and 3), and output layers (cortical layers 5 and 6).
The input layer receives information from the senses
via the thalamus , and the output neurons send motor
and other control outputs to a wide range of subcorti-
cal areas. The hidden layer serves to mediate the output
of the network in response to the input signal by means
of transformations that provide a useful and typically
more elaborated or processed basis for driving the out-
puts of the network. There is a large amount of intercon-
nectivity between the layers, but various forms of data
support the idea that information flows primarily from
input to hidden to output via excitatory neurons. Fur-
ther, these excitatory neurons are bidirectionally con-
nected , so that information can also flow backwards
along these pathways. Inhibitory neurons exist in all
of the cortical layers, and they receive the same types
of excitatory inputs as the excitatory neurons in these
layers. However, they do not send their outputs very far
— they typically inhibit a number of excitatory neurons
within a relatively close proximity to themselves. Thus,
inhibition appears to provide a kind of local feedback
or regulatory mechanism.

Excitatory Interactions

Excitatory neurons can connect with other excitatory
neurons in several distinct ways. A unidirectional or