Memory - Computational Explorations in Cognitive Neuroscience - page 295

Monitor Speakers Keyboard

synthesizer

key−
board

spea−
ker

Hidden

mon−
itor

Input

Figure 9.17: Three related items (television, terminal, syn-
thesizer) represented in an overlapping, distributed fashion as
two out of three common features.

Figure 9.18: Network for exploring active maintenance.

trial Event cycle Input

Hidden

0 Input 9

basins are necessary. These wider basins are present in
the final configuration we explore, where the units are
completely isolated from each other. This completely
prevents activation spread and yields very robust active
maintenance (O'Reilly et al., 1999a), but at the loss of
the ability to perform inference via activation spread.
To explore these ideas, we use the simple overlap-
ping distributed representations of a television , synthe-
sizer ,and terminal in terms of the features of keyboard ,
speaker ,and monitor , as shown in figure 9.17. This
is the same example that we explored in chapter 3, in
the context of a similar kind of problem with activa-
tion spread in distributed representations. Note that in
this earlier case, the problem was solved by using inhi-
bition. That will not help here because, unlike before,
there will be no external inputs to favor the activation of
the appropriate units.

0 Input 19

. . .

0 Input 89

0 Input 99

1 Maintain 9

1 Maintain 19

1 Maintain 29

. . .

1 Maintain 89

1 Maintain 99

Figure 9.19: Grid log showing spreading activation and loss
of maintained information with distributed interconnectivity
among the hidden units.

You should see the bidirectional excitatory connec-
tions among the three hidden units, which in theory
might enable them to actively maintain the representa-
tions even after the input pattern is turned off.
There are two “events” in the environment, one where
an input pattern is presented to the network, and another
where the input is zeroed out (not presented). We are in-
terested in how well the information in the hidden units
is maintained during this second event.

Exploring the Model

Open the project act_maint.proj.gz in
chapter_9 to begin.
You will see that the network has three hidden units
representing the three features (figure 9.18). The input
units provide individual input to the corresponding hid-
den unit. In an active maintenance context, one needs to
have the individual features of a distributed representa-
tion mutually support each other via excitatory connec-
tions.

Return to viewing act in the network, and then
press the Run button.
You will see the network presented with inputs and
the units respond, but it will probably be too quick to
get a clear idea of what happened.

View r.wt in the network window and select the
different hidden units to view their weights.

Do View , GRID_LOG , and then Run again.