Memory - Computational Explorations in Cognitive Neuroscience - page 280

Another thing we can do to improve performance is
to enhance the contribution of the list context inputs rel-
ative to the A stimulus, because this list context disam-
biguates the two different associates.

between runs with default and these new parameters).
Thus, we could play with the lrate parameter to see
if we can speed up learning in the network.

Keeping the same “optimal” parameters, set the lrate
to .1, and do a Batch .
Although the increase in learning rate successfully
speeded up the learning process, repeated testing of this
faster versus slower learning rates shows that the faster
learning rate produces substantially more interference!
This makes sense, because the larger learning rate pro-
duces larger weight changes that undo the AB list learn-
ing faster on the AC list. This should help you appreci-
ate how impressive the fast, relatively interference-free
human performance is.
We will see in the next section that achieving the
level of separation necessary to learn rapidly requires
a specialized neural architecture — our simple cortical
model is just not up to the task.

Do this by changing fm_context from 1 to 1.5.
This increases the weight scaling for context inputs
(i.e., by increasing the r k ( wt_scale.rel ) parame-
ter in equation 2.17). We might imagine that strategic
focusing of attention by the subject accomplishes some-
thing like this.
Finally, increased amounts of Hebbian learning
might contribute to better performance because of the
strong correlation of all items on a given list with the
associated list context representation, which should be
emphasized by Hebbian learning. This could lead to
different subsets of hidden units representing the items
on the two lists because of the different context repre-
sentations.

, !

, !

Do this by setting hebb to .05 instead of .01. Go to the PDP++Root window. To continue on to

the next simulation, close this project first by selecting
.projects/Remove/Project_0 . Or, if you wish to
stop now, quit by selecting Object/Quit .

Clear the graph log and run a Batch with all of
these new parameters.

Question 9.4 (a) Report the average testing error
( avg_tst_se ) for the batch run, and the number of
epochs it takes the network to reach its maximum error
on the AB list after the introduction of the AC list. (b)
Informal testing has shown that this is basically the best
performance that can be obtained in this network — is
it now a good model of human performance?

Summary and Discussion

We have seen that we can improve the rapid, arbitrary
learning performance of a simple cortical network by
moving away from distributed, overlapping representa-
tions toward sparser, separated representations. How-
ever, what we ended up with in the above exploration
was really an awkward compromise network that was
trying to learn sparse, separated representations, but
was unable to achieve sufficient levels of pattern sep-
aration to really work effectively. Thus, this simple cor-
tical architecture seems to work best with the slow, in-
terleaved learning strategy that is so effective for most
other forms of cortical learning.
Instead of trying to achieve all forms of learning
within the cortical system, the brain appears to have
developed two specialized systems (McClelland et al.,
1995). One system, the posterior cortex, uses slow
interleaved learning, and produces the kinds of pow-
erful overlapping distributed representations that we
have discussed and explored throughout the text. The
other system, the hippocampus, uses an extreme form

Although the final level of interference on the AB list
remains relatively high, you should have observed that
these manipulations have significantly slowed the on-
set of this interference. Thus, we have some indication
that these manipulations are having an effect in the right
direction, providing some support for the principle of
using sparse, non-overlapping representations to avoid
interference.
One important dimension that we have not yet em-
phasized is the speed with which the network learns —
it is clearly not learning as fast (in terms of number of
exposures to the list items) as human subjects do. Fur-
ther, the manipulations we have made to improve inter-
ference performance have resulted in even longer train-
ing times (you can see this if you don't clear the log