Error-Driven Task Learning - Computational Explorations in Cognitive Neuroscience - page 166

people typically assume when they think of error-driven
learning, where a teaching signal is explicitly provided
(e.g., a student misreads a word, and the teacher explic-
itly corrects the student). Figure 5.12b shows how sim-
ilar kinds of error signals can arise as a result of making
an implicit expectation (e.g., about how a word should
be pronounced), followed by the actual experience of
hearing the word pronounced (e.g., by one's parent dur-
ing bedtime reading). Figures 5.12c and d show dif-
ferent contexts in which these implicit expectations can
be generated, including expecting the outcome of mo-
tor output, and then experiencing the actual outcome,
or even “expecting” the actual input that one just re-
ceived, which amounts to a kind of generative model
(cf. Dayan et al., 1995).
Another form of implicit error signal (not shown in
the figure) can be produced by the mismatch between
different sensory representations of the same underly-
ing event (Becker, 1996; de Sa & Ballard, 1998; Kay,
Floreano, & Phillips, 1998). Thus, each modality cre-
ates an expectation about how the other modality will
represent the event, and the difference between this ex-
pectation and how the modality actually represents the
event is an error signal for training both modalities.
In addition to the multitude of ways that error sig-
nals could be based on expectation-outcome differ-
ences, ERP recordings of electrical activity over the
scalp during behavioral tasks indicate that cortical ac-
tivation states reflect expectations and are sensitive to
differential outcomes. For example, the widely-studied
P300 wave, a positive-going wave that occurs around
300 msec after stimulus onset, is considered to mea-
sure a violation of subjective expectancy determined by
preceding experience over both the short and long term
(Hillyard & Picton, 1987). Thus, one could interpret the
P300 as reflecting a plus phase wave of activation fol-
lowing in a relatively short time-frame the activation of
minus phase expectations. Although the specific prop-
erties of the P300 itself might be due to specialized neu-
ral mechanisms for monitoring discrepancies between
expectations and outcomes, its presence suggests the
possibility that neurons in the mammalian neocortex ex-
perience two states of activation in relatively rapid suc-
cession, one corresponding to expectation and the other
corresponding to outcome.

Further evidence for expectation-outcome states in
the cortex comes from the updating of spatial repre-
sentations in the parietal cortex as a function of eye
movements (saccades) — neurons actually first antic-
ipate what their new input coding will be as a function
of the motor plan, and then the same neurons update to
reflect their actual input coding as a result of executing
the motor plan (Colby, Duhamel, & Goldberg, 1996).
Specifically, these parietal neurons first represent what
they will “see” after an eye movement before the eye
movement is executed , and then update to reflect what
they actually see after the eye movement. When these
anticipatory and actual representations are discrepant,
there would be just the kind of expectation-outcome
difference that could be used for learning.
Although these ideas and data support the idea that
the brain can learn from error signals using phase-like
differences that occur naturally in many situations, a
number of details remain to be resolved. For exam-
ple, our models assume that the input and output lay-
ers are physically distinct, and that the output under-
goes a minus-plus phase transition while the input re-
mains constant. Although this is reasonable for many
types of error signals (e.g., reading a word, forming
an auditory expectation of its pronunciation, followed
by hearing the actual pronunciation), other types of er-
ror signals require an expectation to be formed within
the same modality as the input that triggered the ex-
pectation (e.g., seeing a block about to fall off a ledge,
and forming the expectation of what will happen next).
In this case, the same set of perceptual units need to
serve as both input and “output” (i.e., expectation) units.
Preliminary explorations of this kind of learning sug-
gest that the same learning mechanism apply (O'Reilly,
1996b), but such issues remain to be systematically ad-
dressed. See chapter 12 for more discussion.

5.8.3 Synaptic Modification Mechanisms

Having suggested that the minus and plus phase activa-
tions follow each other in rapid succession, it remains
to be shown how these two activation states could influ-
ence synaptic modification in a manner largely consis-
tent with the CHL version of GeneRec (equation 5.39).
We will see that the biological mechanisms for LTP/D