Conclusions - Computational Explorations in Cognitive Neuroscience - page 404

of individual detectors to compute a balance of excita-
tory and inhibitory inputs, and include: transforming
input patterns into patterns that emphasize some dis-
tinctions and collapse across others; bidirectional top-
down and bottom-up processing, pattern completion,
amplification, and bootstrapping; attractor dynamics;
inhibitory competition and activity regulation, leading
to sparse distributed representations; and multiple con-
straint satisfaction, where the network's activation up-
dates attempt to satisfy as many constraints as possible.
Chapters 4-6 built on the basic network properties
by showing how neurons within a network can learn by
adapting their weights according to activation values of
the sending and receiving units. Such learning uses lo-
cal variables, yet results in coherent and beneficial ef-
fects for the entire network. We analyzed two main
learning objectives: model learning and task learning.
Model learning causes the network to capture impor-
tant aspects of the correlational structure of the envi-
ronment. Task learning enables the network to learn
specific tasks. These learning objectives are comple-
mentary and synergistic, and can both be achieved us-
ing known properties of synaptic modification mecha-
nisms. Extra mechanisms, including an internal context
representation and a mechanism for predicting future
rewards, can also be used to learn temporally extended
or sequential tasks.
In chapter 7, we sketched an overall cognitive archi-
tecture that builds on the mechanisms just described.
Three different brain areas can be defined on the ba-
sis of fundamental tradeoffs that arise from basic neu-
ral network mechanisms. One tradeoff captures the dis-
tinction between the hippocampal system and the rest
of cortex in terms of learning rate — one system can-
not both learn arbitrary things rapidly and also extract
the underlying regularities of the environment. Another
tradeoff captures the difference between the prefrontal
cortex and the posterior cortex in terms of the ability
to perform rapidly updatable yet robust active main-
tenance. Active maintenance suffers when representa-
tions are richly interconnected, because spreading acti-
vation bleeds away memory. But these interconnections
are useful for many kinds of processing (e.g., pattern
completion, constraint-satisfaction, and inference more
generally). The posterior cortex can thus be understood

in terms of its relationship to these other specialized
brain areas. More generally, the posterior cortex can
be seen as a set of specialized but interacting, hierar-
chically organized pathways of transformations build-
ing upon each other.
Chapter 8 provided a good example of a hierarchi-
cally organized sequence of transformations that lead
to the ability to recognize objects in a spatially invari-
ant fashion. At each level, units transformed the input
by both combining features and integrating over differ-
ent locations and sizes. At the lowest level of corti-
cal visual processing, neurons encode the correlational
structure present in visual images. Although the object
recognition model could recognize individual objects
quite well, it got confused when multiple objects were
present. Adding spatial representations that interacted
with this object processing pathway enabled the sys-
tem to sequentially process multiple objects, and also
accounted for effects of lesions in the spatial process-
ing pathway on performance in the Posner spatial cuing
task.
Chapter 9 explored two different ways that neural
networks can implement memories — in weights and
activations. We saw how priming tasks might tap corti-
cal memories taking the form of small weight changes
produced by gradual learning, or residual activation in
the network. However, we saw that the basic corti-
cal model fell short of capturing other human memory
abilities, consistent with our discussion of fundamental
computational tradeoffs in chapter 7. First, a basic cor-
tical model like those used for learning regularities in
the perceptual domain performs badly when required to
rapidly learn and retain novel information — it suffers
from catastrophic interference. The hippocampus, us-
ing very sparse, pattern separated representations, can
avoid this interference while learning rapidly. Second,
the basic cortical model cannot both be rapidly updated
and robustly maintain activation, and spreading activa-
tion via overlapping distributed representations limit its
maintenance abilities. The prefrontal cortex overcomes
these limitations by having an adaptive, dynamic gat-
ing mechanism and relatively isolated representations.
Finally, the interaction between activation and weight-
based memory can be complex, as we saw in a model
of the A-not-B task.