Error-Driven Task Learning - Computational Explorations in Cognitive Neuroscience - page 168

In light of this proposed mechanism, one can inter-
pret the findings that repeated trains of low-frequency
pulses (e.g., 1 Hz) of activation produce LTD (Bear &
Malenka, 1994) as constituting a series of minus phase
level activations that are not persistent enough to pro-
duce LTP, and thus result in just the kind of LTD that is
predicted by the error correction case.
The proposed mechanism may also interact with an
additional mechanism that indicates when activations
should be considered to be in the plus phase, and thus
when learning should occur based on the level of cal-
cium at that point. This would reduce the need for
the assumption that the minus phase is relatively tran-
sient compared to the plus phase. It seems plausible
that such a plus-phase signal could be produced by
the same kinds of dopamine-releasing mechanisms that
have been described by Schultz, Apicella, and Ljung-
berg (1993) and modeled by Montague, Dayan, and Se-
jnowski (1996) (we will discuss this more in section 6.7
in the next chapter). These midbrain dopamine systems
apparently fire whenever there is a mismatch between
expectation and outcome (specifically in the case of re-
ward, which is what has been studied, but it might be
more general than that). It is also known that dopamine
can modulate the efficacy of LTP, which is appropriate
for a plus-phase like “learn now” signal.
In summary, the available data are consistent with
a biological mechanism that would enable error-driven
task learning, but much more work would need to be
done to establish this fact conclusively and to illuminate
its nature and dynamics more precisely.

ception is that we have introduced a hidden layer of 3
units. Note that there are only feedforward connections
from the input to this hidden layer, because the input is
clamped in both minus and plus phases and so would
not be affected by feedback connections anyway, but
that there are bidirectional connections between the hid-
den and output layers, as required by the GeneRec al-
gorithm. The minor exception is that we have increased
the learning rate from .01 to .05 so that it takes less time
to solve the “impossible” problem. Let's start by giv-
ing the network the hardest problem. By default, the
learn_rule is set to GENEREC ,and env_type is
IMPOSSIBLE .

View the TRAIN_GRAPH_LOG and

the

TEST_GRID_LOG , and then press Run .
As before, the train graph log displays the SSE error
measure over epochs of training. The testing grid log is
now updated only after every 10 epochs of training, and
it also shows the states of the 3 hidden units.
As before, the training of the network stops automat-
ically after it gets the entire training set correct 5 epochs
in a row. Note that this 5 correct repetitions criterion fil-
ters out the occasional spurious solutions that can hap-
pen due to the somewhat noisy behavior of the network
during learning, as evidenced by the jagged shape of
the learning curve. The reason for this noisy behavior is
that a relatively small change in the weight can lead to
large overall changes in the network's behavior due to
the bidirectional activation dynamics, which produces a
range of different responses to the input patterns.
This sensitivity of the network is a property of all
attractor networks (i.e., networks having bidirectional
connectivity), but is not typical of feedforward net-
works. Thus, a feedforward backpropagation network
learning this same task will have a smooth, monotoni-
cally decreasing learning curve. Some people have crit-
icized the nature of learning in attractor networks be-
cause they do not share the smoothness of backpropa-
gation. However, we find the benefits of bidirectional
connectivity and attractor dynamics to far outweigh the
aesthetics of the learning curve. Furthermore, larger
networks exhibit smoother learning, because they have
more “mass” and are thus less sensitive to small weight
changes.

, !

5.9 Exploration of GeneRec-Based Task Learning

Now, let's put some of this theory to work and see how
GeneRec does on some small-scale task learning prob-
lems. We will use the same problems used in the pat-
tern associator case, but we add an extra hidden layer
of units between the inputs and outputs. This should
in theory enable the network to solve the “impossible”
task from before.

Open the project generec.proj.gz in

chapter_5 to begin.
This project is identical to the pattern associator one,
with one major and one minor exception. The major ex-

, !