Memory - Computational Explorations in Cognitive Neuroscience - page 301

Trial Input Maint Output

have also provided purely computational motivations in
favor of such a gating mechanism.
One consequence of this DA regulation idea is that
the VTA should exhibit a burst of (phasic) firing at those
times when the frontal cortex needs to be updated.
Schultz et al. (1993) have found that indeed, the VTA
exhibits transient, stimulus-locked activity in response
to stimuli that predicted subsequent meaningful events
(e.g., reward or other cues that then predict reward).
Further, this role of DA as a gating mechanism is syner-
gistic with its role in reward-based learning, as explored
in chapter 6. As we will see in chapter 11, this kind of
learning mechanism enables the network to adaptively
control the updating and maintenance of working mem-
ory representations as a function of task demands. This
avoids the apparent “homunculus” (small “man” inside
the brain) that would otherwise be required to control
working memory in a useful fashion.
Although we focus on the role of the VTA in this
model, it is also possible that the basal ganglia play
an important role in controlling frontal active memory.
As discussed in chapter 7, the basal ganglia appear to
be specialized for high-threshold detection and initia-
tion of “actions.” In the cognitive domain, the action
that basal ganglia firing may initiate is the gating in
of memories in the frontal cortex. This could be ef-
fected by the disinhibition of cortico-thalamic loops via
the connections from the basal ganglia to the regions
of the thalamus that are interconnected with the frontal
cortex. One advantage of the basal-ganglia system over
the VTA is its potential for much finer resolution in de-
termining which regions of frontal cortex to update, and
which not to update. Although we are currently explor-
ing these ideas, the present model focuses on the VTA
because it is much simpler and the potential advantages
of the basal ganglia system are not relevant to the simple
task explored here.

1 ST E-AA A

2 IGNORE-B A B

3 IGNORE-C A C

4 IGNORE-D A D

5 E LL A A

Tab le 9 . 2 : A sequence of trials in the simple active mainte-
nance task modeled in this section, showing the input (control
cue and stimulus), what should be maintained in active mem-
ory, and what should be output.

the strength of the connections in the prefrontal cor-
tex, and we make the prefrontal representations isolated
and self-maintaining (as we did in a previous simula-
tion). Because the dopaminergic gating system is driven
by the reinforcement learning mechanisms developed
in chapter 6, the model instantiates the important syn-
ergy between reinforcement learning mechanisms and
the learned control of active maintenance.
As discussed in chapter 6, we can think of the pre-
frontal cortex as playing the role of a context represen-
tation in a simple recurrent network (SRN), which is
consistent with the general view of the prefrontal cortex
as providing an internal representation of task context
(Cohen & Servan-Schreiber, 1992). When we add in the
dynamic dopamine-based gating mechanism, this con-
text representation can become much more flexible than
that of a standard SRN, which always performs a simple
copy operation after each trial. To harness this flexibil-
ity, the gating mechanism must learn when to update
working memory — this is where the dopamine-based
reinforcement learning mechanisms discussed in chap-
ter 6 are used. If you have not yet read section 6.7 you
might want to do so now before continuing — it will
enable you to understand the model in greater detail.
Otherwise, you should still be able to get the main ideas
from the presentation here.
To explore the basic principles of active maintenance
in the model, we will use a simple task that requires
information to be stored and maintained over time in
the face of other distracting inputs, and then recalled.
These are the basic task requirements of the working
memory span tasks commonly used to measure work-
ing memory capacity

9.5.2 Details of the Prefrontal Cortex Model

The model we will explore in this section incorporates
the specializations discussed earlier, enabling us to sim-
ulate the role of the prefrontal cortex in active mainte-
nance. Specifically, we include a dynamic gating mech-
anism (based on the dopamine system) that regulates

(Daneman & Carpenter, 1980;