Information Technology Reference
In-Depth Information
Output
Exploring the Model
The basic framework for implementing the AB-AC task
is to have two input patterns, one that represents the A
stimulus, and the other that represents the “list context”
(figure 9.5). Thus, we assume that the subject devel-
ops some internal representation that identifies the two
different lists, and that this serves as a means of dis-
ambiguating which of the two associates should be pro-
duced. These input patterns feed into a hidden layer,
which then produces an output pattern corresponding to
the B or C associate. As in the previous simulation, we
use a distributed representation of random bit patterns
to represent the word stimuli.
Hidden
Input
Context
Figure 9.5: Network for the AB-AC list learning task, with
the Input layer representing the A stimulus, the Context
input representing the list context, and the Output being the
B or C word associate, depending on the list context.
Open ab_ac_interference.proj.gz in
chapter_9 to begin.
You will see the network as described previously.
First, let's look at the training environment.
Press the View , EVENTS_AB button to view the dis-
tributed input/output patterns for the AB list. Do View ,
EVENTS_AC to see the AC items.
The bottom patterns for each event are the A item and
the list context, and the upper pattern is the associate.
Note that the first item in the AB list ( 0_b ) has the same
A pattern as the first item in the AC list ( 0_c ). This is
true for each corresponding pair of items in the two lists.
The list context patterns for all the items on the AB list
are all similar random variations of a common under-
lying pattern, and likewise for the AC items. Thus, the
list context patterns are not identical for each item on
the same list, just very similar (this reflects the fact that
even if the external environmental context is constant,
the internal perception of it fluctuates, and other inter-
nal context factors like a sense of time passing change
as well). Go ahead and iconify these windows.
Now, let's see how well the network performs on this
task.
long-term priming network from the previous section).
Although the second option is compatible with our ba-
sic cortical model, it does not address the situation that
we are interested in here, where one needs to rapidly
and sequentially learn novel information. Therefore,
we will explore the first option. However, as we have
stressed previously, this solution conflicts with the use
of overlapping distributed representations, where the
same units participate in the representation of many dif-
ferent items.
We will see that we can either have a set of param-
eters that result in overlapping distributed representa-
tions (in which case we get catastrophic interference),
or we can have a set of parameters which result in
sparser (having fewer units active) separated represen-
tations, where we can reduce interference, but give up
the benefits of overlapping distributed representations.
In section 9.3 below, we will explore a model of the hip-
pocampus (and related structures), which appears to re-
solve this dilemma by providing a specialized memory
system based on sparse separated representations that
can subserve rapid learning without suffering undue in-
terference.
Do View , TRAIN_GRAPH_LOG to get a graph log
of training performance.
Then, do Run on the control
panel.
You will see the graph log being updated as the net-
work is trained, initially on the AB list, and then on
the AC list. The red line shows the error expressed as
the number of list items incorrectly produced, with the
criterion for correct performance being all units on the
Search WWH ::




Custom Search