Networks of Neurons - Computational Explorations in Cognitive Neuroscience - page 80

changing the units' excitability (via the leak current) af-
fect the success of the network, and why?

.projects/Remove/Project_0 . Or, if you wish to
stop now, quit by selecting Object/Quit .

Letter Inputs 3.3.2 Localist versus Distributed Representations

The digit network we explored in the previous section
used very simple detectors with weight patterns that di-
rectly matched the input patterns they were intended to
detect, often referred to as template matching repre-
sentations. Another aspect of these units was that only
one unit represented each input pattern, referred to as a
local or localist representation, which often goes hand
in hand with template matching ideas. Such representa-
tions are also often referred to as grandmother cell rep-
resentations, because their use must imply that some-
where in the brain is a neuron that (uniquely) represents
one's grandmother.
Though useful for demonstration purposes, localist
representations are not particularly realistic or power-
ful. Thus, it is unfortunate that the detector model of the
neuron is often associated with this type of representa-
tion — as we made clear in the previous chapter, the
detector function can apply to very complex and diffi-
cult to describe nonlocalist contents or representations.
Recordings of cortical neurons have consistently
shown that the brain uses distributed representations,
where each neuron responds to a variety of different
input stimuli, and that many neurons are active for
each input stimulus. For example, researchers have
presented visual stimuli that are systematically varied
along a number of different dimensions (size, shape,
color, etc.), and recorded the activities of neurons in
the areas of the cortex that process visual inputs (see
Desimone & Ungerleider, 1989; Tanaka, 1996, for re-
views). In all cases that we know, these studies have
shown that cortical neurons exhibit measurable tuning
curves , which means that they respond in a graded fash-
ion to stimuli within a range of different parameter val-
ues (figure 3.12). Furthermore, it is clear both from the
overlap in these tuning curves and from multiple paral-
lel recording that many neurons are simultaneously en-
coding the same input stimuli.
It is often useful to think about distributed units as
representing features of input stimuli, where a given
stimulus is represented by a collection of units that each

Now, we will see how the network responds to letter
inputs instead of digits.

First, set the g_bar_l leak conductance back to
6(or Defaults ) and make sure act is selected in
the NetView window. Then set env_type to LETTERS
( Apply ) and press Run .
Notice the letters being presented over the input layer
on the network.

, !

Use the VCR-like controls at the top of the Grid-
Log to scroll the display back to the start of the letter
presentations, because these have scrolled off the “top”
of the display (the single < button is a good choice for
fine-grained scrolling).
The only significant response came to the “S” letter
input from the “8” hidden unit — note that “S” is very
similar to the “8”.

Press the fast-forward-to-the-end button ( >| )on
the grid log, so that it will continue to scroll when you
next Run it. Press Cluster , CLUSTER_LETTERS .Next,
press Cluster and CLUSTER_HIDDEN .
The first cluster plot should look like figure 3.10a,
and the second should look like figure 3.10b. You
should be able to see in these cluster plots that these
digit units do not respond very informatively to the let-
ter stimuli.

, !

Question 3.4 (a) Based on your experiences in the pre-
vious question, what would you expect to happen to the
cluster plot of hidden responses to letter inputs as you
lowered the g_bar_l leak current to a value of 4? Do
this — were you right? (b) Would you say that this
hidden representation is a good one for conveying letter
identity information? Why or why not? (Hint: Pay par-
ticular attention to whether any letters are collapsed in
the cluster plot — i.e., having no distance at all between
them.) (c) Can you find any setting of g_bar_l that
gives you a satisfactory hidden representation of letter
information? Explain.

Go to the PDP++Root window. To continue on to
the next simulation, close this project first by selecting

, !