Memory - Computational Explorations in Cognitive Neuroscience - page 304

when reward is delivered. The problem with this solu-
tion is that network-based active maintenance mecha-
nisms (e.g., recurrent activation) cannot easily tell the
difference between representations that are transiently
active and those that should be maintained — once a
set of units becomes active, recurrent connectivity will
automatically maintain them.
One attractive solution to this problem would be if
there were a biophysical switch where an additional
excitatory current is activated for those neurons that
should be maintaining information, but not for the tran-
siently active neurons. When the input signals go away,
those neurons with this extra current will dominate and
thereby maintain information. Dopamine could then
play a role in throwing this switch. Interestingly, there
is some evidence for exactly this set of mechanisms in
the PFC (Yang & Seamans, 1996) and considerably
more evidence in the basal ganglia (e.g., Surmeier,
Baras, H. C. Hemmings, Narin, & Greengard, 1995;
Surmeier & Kitai, 1999), with several computational
models exploring this kind of mechanism (Camperi &
Wang, 1997; Fellous, Wang, & Lisman, 1998; Dilmore,
Gutkin, & Ermentrout, 1999). Furthermore, frontal
recordings have shown that inputs transiently control
neural firing there, but that the maintained activation
pattern subsequently re-emerges (Miller et al., 1996).
We have implemented the present model using the
intrinsic current switch mechanisms, and although it
works better in some respects than the recurrent-
activation based model, these mechanisms are some-
what more complex and less contiguous with previous
models. Thus, we explore the version where mainte-
nance depends entirely on recurrent activation here.

the connections that will enable it to predict subsequent
reward. Also, the PFC receives a “dummy” connection
from the AC — this does not actually send any activa-
tion, and is there to enable the PFC to read the Æ signal
from the AC unit, which is used to set the gating modu-
lation of the PFC connections. The second hidden layer
( Hidden2 ) receives from the first hidden layer and the
PFC, and is trained by standard error-driven (and Heb-
bian) learning to produce the appropriate output on the
output layer.
Now, let's step through some trials to see how the
task works.

Switch back to viewing activations ( act ). Do Step
in the pfc_ctrl control panel.
In the input, you can see that the store input cue unit
(labeled S ) and one of the stimuli units are activated.
This activation has spread to the corresponding hidden
layer units, and the Hidden2 layer has computed a
guess as to the correct output (at random at this point).
Thus, this is a standard minus activation phase. As ex-
plained in chapter 6, the AC unit is actually clamped to
its prior value in the minus phase. Similarly, we also
clamp the PFC representations to their prior values in
the minus phase. Thus, these are both zero at this point.

Press Step again.
The subsequent plus phase happens, where the cor-
rect output value is presented. It is also during this time
that the AC unit is free to settle into whatever activa-
tion state its inputs dictate, or if an external reward was
provided on this trial, then the AC unit would instead
be clamped to that reward value. Because the proper
setting of the modulation of the PFC weights depends
on the difference between the plus and minus activation
states of the AC unit, we then have to run a second plus
phase where the PFC units are updated with the gain
parameters set appropriately. In reality, these two plus
phases need not be so discretely separated, and can in-
stead be viewed as two aspects of a single plus phase.

, !

9.5.3 Exploring the Model

Open the project pfc_maint_updt.proj.gz in
chapter_9 to begin.
You should see the network depicted in figure 9.21.

, !

Press Step again to see the second plus phase.
Nothing will happen, because there was no reward,
and so the difference in AC activations was 0, and thus
the modulation of the weights going into the PFC is at
the base level (which is 0 in this case). However, as we
will see, sometimes the gating noise ￿ will be sufficient
to allow the PFC units to become activated.

As usual, use r.wt to view the network connectivity.
You should see that the hidden and prefrontal cor-
tex (PFC) layers have one-to-one connectivity, and the
PFC has isolated self-connectivity to enable it to main-
tain information without activation spread. The AC unit
receives from the hidden and PFC layers — these are