Memory - Computational Explorations in Cognitive Neuroscience - page 306

9.5.4 Summary and Discussion

Do View , GRID_LOG , and return to viewing act in
the network.
You should see a grid log that shows the activations
of the main layers in the network for each phase of pro-
cessing.

This model has shown how there can be an important
synergy between reinforcement learning mechanisms
and the learned control of active maintenance. The
model brings together biological data on the role of
dopamine in the prefrontal cortex with computational
constraints that motivate the need for dynamic control
over updating and maintenance in the active working
memory system. That there appears to be a good syn-
ergy between these different levels of analysis is en-
couraging, but definitive support for these ideas is not
yet available.
Another source of encouragement for these ideas
comes from the work of Braver and Cohen (2000),
who present a model of the AX-CPT (continuous per-
formance task) based on largely the same principles as
the model just we explored. This AX-CPT task in-
volves maintaining stimulus information over sequen-
tial trials to detect certain target sequences (e.g., A fol-
lowed by X) as distinguished from other non-target se-
quences (e.g., A followed by Y, or B followed by X,
or B followed by Y). This task has been extensively
studied in behavioral and neuroimaging studies (e.g.,
Servan-Schreiber et al., 1997; Barch, Braver, Nystrom,
Forman, Noll, & Cohen, 1997), and the model explains
a number of patterns in the data in terms of dynamically
controlled active memory mechanisms.
Although many of the ideas regarding the way the
prefrontal cortex is specialized are speculative and go
beyond existing data, they point to the importance of
using a computational approach toward understanding
brain function. The computational models we have ex-
plored have highlighted a set of constraints that the pre-
frontal cortex likely must satisfy in one way or another.
This provides a unique source of insight into the role
of specific aspects of the biology of the frontal cortex
(e.g., why it has such an intimate relationship with the
dopamine system), and into the role of the frontal cortex
within the larger cognitive system (e.g., by contrasting
the role of spreading activation in the frontal and the
posterior cortex). In short, we think the underlying mo-
tivations and computational constraints we have identi-
fied will outlive any specific details that may not hold
up over time.

Press Clear on this grid log. Then, Step through
an entire store-ignore-recall sequence.
You should observe that the activation pattern on the
store trial (the S unit and the stimulus) are stored in ac-
tive memory in the PFC, and maintained in the face of
the intervening “ignore” trials, such that the network is
able to produce the correct output on the recall trial.

, !

Question 9.12 (a) Describe what happens to the AC
unit on the store trial, and then throughout the remain-
der of the sequence. (b) Explain how the AC unit accu-
rately predicts future reward, and at what point it does
so (note that the external reward is visible as the acti-
vation state of the AC unit on the first plus phase of the
recall trial).

You might have also noticed that the hidden units cor-
responding to those active in the PFC were not strongly
activated on the correct recall trial, even though we have
a mechanism for causing the PFC units to activate the
hidden units to avoid the catch-22 problem described
earlier. Recall that these output weights from the PFC
are only activated by a positive AC phase difference
(temporal differences) signal — when the network ac-
curately predicts reward, this signal is 0, and therefore
these weights are not enhanced, and the hidden units do
not reflect the PFC activation. Thus, this model makes
an interesting prediction regarding the reactivation of
information maintained in the PFC when the task is be-
ing performed correctly. This is only one of a number of
similar such predictions that this model makes regard-
ing neural activations in various parts of the system as
a function of the progression of learning. Thus, neu-
ral recording data in monkeys performing similar such
tasks could provide important tests of the basic ideas in
this model.

Go to the PDP++Root window. To continue on to
the next simulation, close this project first by selecting
.projects/Remove/Project_0 . Or, if you wish to
stop now, quit by selecting Object/Quit .

, !