Memory - Computational Explorations in Cognitive Neuroscience - page 297

tained information — the active maintenance system
must be capable of overcoming this degradation.

there is no interconnectivity between units, it is impos-
sible for activation to spread to other representations,
resulting in perfect maintenance of information even in
the presence of large amounts of noise. These isolated
units can be self-maintaining by having an excitatory
self-connection for each unit.

To add noise (we just add it to the membrane poten-
tial on each time step), set the noise_var parameter
on the control panel to .01. Do several Run s.
You should have observed that in the presence of
noise, even the higher-order distributed representations
cannot prevent the spread of activation. The explana-
tion is relatively straightforward — the noise was suf-
ficiently large to move the network outside of the orig-
inal attractor basin and into that of another represen-
tation. This indicates that the higher-order distributed
representations may not have sufficiently wide attractor
basins for robust active maintenance.
A parameter that should play an important role in this
network is the strength of the recurrent weights. For ex-
ample, if these weights were made sufficiently weak,
one would expect that the network would be incapable
of active maintenance. At the other extreme, it might be
the case that very strong recurrent weights would pro-
duce a more robust form of active maintenance that bet-
ter resists noise. The strength of the recurrent weights is
determined by the wt_scale parameter in the control
panel, which has been set to 1.

To explore this kind of representation, set net_type
to ISOLATED ( Apply ). You can verify the connectivity
by using r.wt .Setthe wt_scale back to 1, but keep
the noise_var at .01, and then Run several times.
You should observe that the network is now able to
maintain the information without any difficulty, even
with the same amount of noise that proved so damag-
ing to the previous network. However, this isolated net-
work no longer has the ability to perform any of the
useful computations that require knowledge of which
features go together, because each unit is isolated from
the others. Nevertheless, the posterior cortex can rep-
resent all of this relationship information via overlap-
ping distributed representations, so it should be okay
for a specialized active maintenance system to use more
isolated representations, given their clear advantages in
terms of robustness. We will explore this idea further in
section 9.5.

, !

Set this now to .05 (and keep the noise set to .01),
and do a couple of Run s.
You should observe that the network is now no longer
capable of maintaining information once the input goes
away.

, !

9.4.3 Robust yet Rapidly Updatable Active

Maintenance

In addition to the basic need for maintaining informa-
tion over time (without the kind of activation spreading
that we saw above), activation-based working memory
representations also need to meet two potentially con-
flicting needs: they sometimes need to be maintained in
the face of ongoing processing, while at other times they
need to be updated as a function of current information.
For example, when doing mental arithmetic, one needs
to maintain some partial products while at the same time
computing and updating others.
The following simple task, which is similar in many
respects to the continuous performance tasks (CPT) of-
ten used to test working memory (Servan-Schreiber,
Cohen, & Steingard, 1997; Rosvold, Mirsky, Sarason,
Bransome, & Beck, 1956), provides a clear demonstra-
tion of working memory demands. Stimuli (e.g., letters)
are presented sequentially over time on a computer dis-

Now let's see if making the recurrent weights
stronger improves the ability to overcome noise. Try
wt_scale values of 2 and 5 with multiple Run s each.

Question 9.10 (a) Does this seem to improve the net-
work's ability to hold onto information over time? (b)
Explain your results, keeping in mind that the recurrent
weights interconnect all of the hidden units.

Although some kinds of distributed representations
could exhibit sufficiently robust active maintenance
abilities, there is another type of representation that is
guaranteed to produce very robust active maintenance.
This type of representation uses isolated units that do
not have distributed patterns of interconnectivity, and
thus that have very wide basins of attraction. Because