Memory - Computational Explorations in Cognitive Neuroscience - page 293

trial
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

Event
0_a
0_b
1_a
1_b
2_a
2_b
3_a
3_b
4_a
4_b
5_a
5_b
6_a
6_b
7_a
7_b
8_a
8_b
9_a
9_b
10_a
10_b
11_a
11_b
12_a
12_b

sum_se
0
1.7529
0
2.18947
0
5.43822
0
1.05335
0
6.26163
0
4.02698
0
5.74102
0
8.85609
0
9.4205
0
7.888
0
5.20613
0
6.4702
0
5.32969

Outp_dist ev_nm

0_a
0_b
1_a
1_a
2_a
2_a
3_a
3_b
4_a
4_a
5_a
5_a
6_a
6_a
7_a
7_a
8_a
8_a
9_a
9_a
10_a
10_a
11_a
11_a
12_a
12_a

sm_nm both_err

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

Change the act_decay parameter to 0 instead of
1. Press Apply and Test again.
You should notice an increased tendency for the net-
work to respond with “a” in the “b” trials (figure 9.15).

0
1.7529
0
2.06997
0
0.467382
0
1.05335
0
0.663053
0
2.36882
0
2.00435
0
0
0
0.444151
0
1.64196
0
0.337607
0
1.40431
0
0.33391

0
0
0
1
0
1
0
0
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1

Question 9.8 (a) Report the number of times the net-
work responded “a” instead of “b” for the “b” test tri-
als. (b) Explain why act_decay has this effect on
performance.

In summary, this simulation replicates our earlier
weight-based priming results using residual activations
instead of weight changes, and illustrates a simple form
of activation-based memory that our model of the cortex
naturally exhibits.

Go to the PDP++Root window. To continue on to
the next simulation, close this project first by selecting
.projects/Remove/Project_0 . Or, if you wish to
stop now, quit by selecting Object/Quit .

Figure 9.15: Text log output showing activation-based prim-
ing effects — the network responds with the a outputs (shown
in the ev nm column) to the b inputs (shown in the Event
column), as a result of the immediately prior input.

9.4.2 Active Maintenance

In addition to simple priming, it would be useful to
have a form of activation-based memory that persists
for longer periods of time, and that can maintain infor-
mation even in the face of noise or interference from
activity elsewhere in the network. This kind of mem-
ory would be useful for things like holding on to in-
formation that is needed for ongoing processing, as in
the case of mental arithmetic as mentioned previously
(e.g., computing 42 ￿ 7 ). In this task, one needs to
keep track of intermediate results and also where one
is in the sequence of steps necessary to solve the prob-
lem. Maintaining this information in an active form
(i.e., active maintenance ) is useful because it needs to
be rapidly updated (i.e., as you move on to subsequent
steps), and it also needs to be readily accessible to on-
going processing — having active representations of the
relevant information achieves these goals in a way that
more passive weight-based memories cannot. This kind
of activation-based memory is typically referred to as
working memory (Baddeley, 1986).
The most obvious neural network mechanism for
achieving active maintenance is recurrent bidirectional
excitatory connectivity, where activation constantly cir-
culates among active units, refreshing and maintaining

Press Test in the control panel.
This test has the “a” and “b” responses to a given
input being presented one after another, which allows
us to determine the immediate impact of seeing the “a”
case on the response to “b.” When the “a” case is pre-
sented, we clamp the output response pattern to the “a”
value. When the “b” case is then presented, only the
input pattern is presented, so the “b” output response
just serves as a comparison pattern for the actual output
produced by the network given the input. You should
observe that the network produces the “a” response for
the “b” trials about half the time. However, this is not an
indication of priming, because we are completely reset-
ting the activations after each event is presented! Thus,
this roughly 50 percent “a” response reflects the random
biases of the trained network.
The act_decay parameter in the
act_prime_ctrl overall control panel con-
trols the extent to which the activations are decayed
(reset) after each event is processed. Let's observe the
effects of keeping the activations completely intact
from one trial to the next.

, !