Conclusions - Computational Explorations in Cognitive Neuroscience - page 406

12.3.1 Models Are Too Simple

The need for and use of simplification is at once one of
the greatest strengths and weaknesses of neural network
modeling. It is a strength because it allows one to ex-
tract, for example, the essential kernel of insight about
which biological properties are important for a particu-
lar behavior, and why. It is a weakness because behind
each simplification lies a largely uncharted sea of de-
tails, any one of which could render the simplification a
blatant misconstrual of the facts.
An example of the importance of simplification
comes from chapter 2, where the neuron was intention-
ally presented as simpler than its biological complexity
might otherwise suggest. To summarize, detailed com-
plexity in the integration of neural inputs, or a complex
encoding of information in the detailed timing of the
spiking output, is inconsistent with several prominent
properties of the brain. For example, the brain is noisy
and needs to be robust, but these complex mechanisms
are brittle and would be too easily disturbed. Neurons
have only one output signal, yet receive thousands of in-
puts — there isn't enough bandwidth for complex pro-
cessing over thousands of inputs to be conveyed in any
useful form through a single output. Furthermore, neu-
rons act collectively and each individual one makes only
a minor, incremental contribution.
This example provides a cautionary statement against
becoming swept away with bottom-up detail — these
details must always be considered within a larger (and
often simplified) functional framework. In this exam-
ple, although certain biological details might suggest
that the neuron is a very complex computational de-
vice, these details must be evaluated in the context of
the overall nature of neural computation, which strongly
supports a simpler, graded conception of the neuron.
Furthermore, simpler things often just work better.
We are reminded of the currently popular technique of
making mosaic images out of a large number of smaller
images that serve as pixels in the larger image. If you
get too close to one of these images, it just looks like
a random ensemble of little images. However, if you
step back and squint your eyes, the larger overall pic-
ture emerges. Thus, it is simpler to describe the overall
image as “A picture of Princess Diana” instead of de-

Cognitive phenomena

Figure 12.1: Ideal relationship between different levels of
modeling, varying in level of detail and in the range of cog-
nitive phenomena covered. If each modeling effort overlaps
to some extent with others at different levels of detail or ad-
dressing other phenomena, then there are beneficial mutual
constraints between models.

scribing the properties and configurations of all the in-
dividual component images. Similarly, it may be that
squinting our eyes and ignoring some of the biological
details produces a simpler, more relevant picture of neu-
ral computation. Obviously as scientists we cannot take
it on faith that this is so — instead we must also labori-
ously assemble the mosaic from its pieces and confirm
that when you put it together right, it really does look
like something simple.
Although individual researchers must face tradeoffs
in deciding what is most important to study at any given
point in time, the larger plurality of the field allows for,
and benefits from, multiple parallel approaches. We see
this as the key to ultimately solving the simplification
problem — many different models at distinct but over-
lapping levels of analysis and detail (figure 12.1). Thus,
where one model makes a simplification, another delves
into further detail. To the extent that the two models
can be compared in their area of overlap, the simplified
model can be either validated or improved by taking
into account the behavior of the more detailed model
(or at least the limitations of the simplified model will
be known). Likewise, the simplified model can point
to the functionally relevant details, and focus the more
detailed model on them.
Examples of the effective use of multiple overlap-
ping models exist throughout this topic. For example, in
chapter 2, we compared the performance of a unit that
fires discrete spikes with one that computes a rate code.
In chapter 3, we compared inhibition using detailed
inhibitory interneurons with the kWTA simplification.