Combined Model and Task Learning, and Other Mechanisms - Computational Explorations in Cognitive Neuroscience - page 187

value between 0 and 1) determines the extent to which
the context unit gets updated by new input from the hid-
den layer, and fm prv (“from previous,” also between
0 and 1, and typically set to 1 ￿ k hid ) determines how
much it reflects its previous state. The ability of the con-
text units to reflect their prior state (introduced by Jor-
dan, 1986) gives them some limited additional capacity
to maintain information beyond the 1 time-step Marko-
vian window. In the basic SRN, fm hid and fm prv are
constants (that we typically set to .7 and .3), but we will
see in chapter 9 that these parameters can be placed un-
der dynamic control by a specialized gating mechanism
to control the update of context.
Appendix B contains a detailed description of how to
construct an SRN context layer in the simulator.

use context appearing immediately before an ambigu-
ous word, but not context from a previous sentence.
The intersection of both of these findings, sequenc-
ing and internal context maintenance, points directly
to something like the SRN context layer, which pro-
vides the context necessary for sequential or temporally
extended tasks. Further, these data (and many more
like them) show that such context representations are
used not only to produce sequential behavior, but also
to comprehend sequentially presented information like
language (see chapter 10 for a model of this).
In addition to neuropsychological/behavioral data on
frontal cortex, there is a lot of evidence that neurons in
this brain area exhibit sustained firing over task-relevant
delays and sequences (e.g., Goldman-Rakic, 1987;
Fuster, 1989; Miller, Erickson, & Desimone, 1996;
Barone & Joseph, 1989). Anatomically, the frontal cor-
tex receives from and projects to most other areas of
posterior (nonfrontal) cortex, so it has the requisite con-
nectivity to both produce appropriate context represen-
tations and have these context representations affect on-
going processing in the posterior system (which is then
considered to be like the hidden layer in figure 6.12). As
we argued previously, it is not necessary that the frontal
context be a literal copy of the posterior cortex.

6.6.2 Possible Biological Bases for Context

Representations

Perhaps the most obvious candidate brain area for
having something like a context representation is the
frontal cortex . As discussed in greater detail in chap-
ters 9 and 11, the frontal cortex (especially the pre-
frontal cortex) seems to be involved in planning and ex-
ecuting temporally extended behaviors. For example,
people with frontal lesions are often incapable of exe-
cuting a sequence of behaviors, even for somewhat rou-
tine tasks such as making a cup of coffee, even though
they can perform each individual step perfectly well.
Thus, they appear to have a specificdeficit in sequenc-
ing these steps.
In addition to sequencing, the frontal cortex appears
to be important for maintaining representations over
time. Cohen and Servan-Schreiber (1992) argued, and
demonstrated with neural network models, that this in-
ternal maintenance system is important for what they
called context . For example, consider ambiguous words
such as pen that require some kind of context to dis-
ambiguate their meaning (e.g., writing implement vs.
fenced enclosure). They argued that the frontal cor-
tex is responsible for maintaining the necessary internal
context to disambiguate these words, where the con-
text is established by information presented earlier in
a text passage. Further, they showed that people with
impairments in frontal functioning (schizophrenics) can

6.6.3 Exploration: Learning the Reber Grammar

A good example of a task illustrating how the SRN
worksisthe Reber grammar task, modeled by Cleere-
mans, Servan-Schreiber, and McClelland (1989). Re-
ber (1967) was one of the first psychologists to inves-
tigate implicit learning capabilities, which he explored
by having people memorize strings of letters that, un-
beknownst to the subject, followed a regular, but proba-
bilistic, grammar. Cleeremans and McClelland (1991)
investigated a version of Reber's implicit learning task
where participants pushed buttons corresponding to let-
ters that appeared on a computer screen. Subjects ended
up pressing the buttons faster and faster for grammati-
cal sequences of letters, but they were not significantly
faster for sequences that did not follow the grammar.
Thus, they had shown evidence of learning this gram-
mar implicitly, but they explicitly had no knowledge of
this grammar.