Information Technology Reference
In-Depth Information
control cues (S, I, and R for store, ignore, and recall,
respectively), and four units for the four stimuli. The
output has the four stimuli units. The hidden (poste-
rior cortex) and prefrontal cortex (PFC) representations
are simple one-to-one copies of the input representa-
tions. The adaptive critic (AC) unit, which represents
the dopamine controlling system, learns to control when
to update and maintain information in the PFC on the
basis of rewards delivered as a function of whether it
produces the correct output on recall trials. Specifi-
cally, a “positive” reward of 1 is provided to the AC if
the originally stored stimulus is produced on the output,
and a “negative” reward of 0 is given otherwise. The
second hidden layer ( Hidden2 ) learns to map the hid-
den and PFC representations to the correct outputs —
it does this using standard error-driven learning (under
the assumption that the correct answer is provided).
The central feature of the model is how it captures the
interaction between dopamine and the prefrontal cor-
tex, based on the dynamics of the adaptive critic unit.
The AC unit learns to predict future reward on the basis
of presently available stimuli using the temporal differ-
ences (TD) algorithm (Sutton, 1988). The change in
activation of this AC unit (i.e., the temporal-difference)
provides a model of the stimulus-driven changes in
dopamine release relative to its constant baseline ( tonic )
level (Schultz et al., 1997; Montague et al., 1996).
Thus, when the AC's activation increases (signaling an
increase in expected future rewards), this corresponds
to an extra pulse of dopamine release, and when its ac-
tivation decreases, there is a decrease or gap in the level
of dopamine release. This AC unit corresponds to the
neural system (which may be located in the prefrontal
cortex itself) that controls the firing of the dopaminergic
neurons in the VTA.
The model uses this dopaminergic signal to modu-
late the strength of the connections in the PFC. Specif-
ically, when there is an increase in dopamine release,
all the connections coming from the posterior cortex
into the PFC are transiently strengthened by a uniform
modulatory gain factor (implemented in the simulator
by dynamically changing the wt_scale.abs param-
eters that govern the strength of different projections,
see section 2.5.1). This will update the representations
in the PFC as a function of the current inputs.
SI RABCD
PFC
AC
S I
R A B C D
Hidden
Hidden2
SI RABCD
ABCD
Input
Output
Figure 9.21: The prefrontal cortex active maintenance
model. The input contains task control units (S=store,
I=ignore, R=recall) and stimulus units (A-D). The output con-
tains the stimulus units. The hidden (posterior cortex) and pre-
frontal cortex (PFC) have simple one-to-one representations
of the input stimuli. The AC unit and the output hidden layer
learn the significance of the cues for task performance, and
how to produce the appropriate outputs.
Shah & Miyake, 1996). In our version, explicit cues
are presented that mark each input as something to be
stored, ignored, or recalled. The network must learn to
use these cues to dynamically control the updating and
maintenance of active memory. There are four stimuli
that we will label A-D, any of which can serve as some-
thing to be maintained and later recalled, or as some-
thing to be ignored. After each stimulus presentation,
the network is required to produce an output, which is
always just the stimulus being presented on the input,
except in a recall trial, where it is the previously stored
stimulus. Thus, the “ignore” stimuli cannot be com-
pletely ignored if a correct output is to be produced on
ignore trials — they are just ignored from the perspec-
tive of the frontal active maintenance system. It is this
need to maintain some information while at the same
time processing other information that lies at the heart
of the notion of working memory as distinct from sim-
ple short-term memory, and is the main thing tested by
the working memory span tasks.
An example sequence of trials in the task is shown
in table 9.2, and figure 9.21 shows the model for this
task.
The input representation has three units for the
Search WWH ::




Custom Search