Error-Driven Task Learning - Computational Explorations in Cognitive Neuroscience - page 165

a particularly powerful form of error-driven learn-
ing would certainly constitute one such evolutionary
pressure.

a) Explicit Teacher b) Implicit Expectation

Hidden Hidden

− − Input Output' Output Input Outcome' Outcome

Individual unit bidirectional connectivity (for which
we are not aware of any specific biological evidence)
is not critical to the proper functioning of CHL, as
long as there are other pathways for error signals to
propagate along (Galland & Hinton, 1991). Specif-
ically, Galland and Hinton (1991) showed that CHL
was still effective even when all of the connectivity
was asymmetric (i.e., for each pair of hidden and
output units, only one of the two possible connec-
tions between them existed). This robustness can be
attributed to redundancy in the ways that error sig-
nal information can be obtained; a given hidden unit
could obtain the error signal directly from the output
units, or indirectly through connections to other hid-
den units.

t t+.5 t+1 t t+.5 t+1

c) Implicit Motor Expectation d) Implicit Reconstruction

Hidden Hidden

− −

Motor Outcome' Outcome Input Input' Input

t t+.5 t+1

t t+.5 t+1

Figure 5.12: Origin of different forms of error signals, for
a simple three-layer network. The right-most layer in each
figure represents the state of the adjacent layer (to the left)
at subsequent time step, not an additional layer. a) Explicit
teaching signal, where an overt output is made, and then cor-
rected. b) Implicit expectation-based error signal where an
input triggers a subsequent expectation, which is then experi-
enced. c) Implicit expectation based on motor output, where
the consequences of the motor action are expected and then
experienced. d) Implicit reconstruction of a single input layer,
like an auto-encoder (note that here there are just two lay-
ers, with the different input boxes representing different states
over time).

Thus, GeneRec actually requires only rough bidirec-
tional connectivity, the existence of which is supported
by biological evidence. Also, the Hebbian learn-
ing mechanism that we will use in conjunction with
GeneRec in Leabra (the next chapter discusses this
combination in detail) is only approximately symmet-
ric, but the use of this form of learning generally im-
proves learning, so it is clear that the symmetry require-
ment is only approximate.

tablishes a subsequent state of activation, which serves
as the plus locomotive phase. 2 This idea that the brain is
constantly generating expectations, and that the discrep-
ancies between these expectations and subsequent out-
comes can be used for error-driven learning, has been
suggested by McClelland (1994) as a psychological
interpretation of the backpropagation learning proce-
dure. It is particularly attractive for the GeneRec/CHL
version of backpropagation, which uses only activation
states, because it requires no additional mechanisms for
providing specific teaching signals other than the effects
of experience on neural activation states via standard ac-
tivation propagation mechanisms.
Figure 5.12 provides some illustrations of how these
kinds of expectation-based signals could arise under
various conditions. F igure 5.12a shows the case that
2 The word “locomotive” here is just to demonstrate that such ex-
pectations are being generated and it is salient when they are violated.

5.8.2 Phase-Based Activations in the Cortex

The phase-based activations that are central to the
GeneRec algorithm raise perhaps the most controver-
sial aspect of error-driven learning in the cortex: Where
does the teaching signal come from? As discussed pre-
viously, we view the teaching signal as just another state
of experience (i.e., an activation state in the network)
that results from the actual outcome of some previous
conditions. Thus, the minus phase can be viewed as the
expectation of the outcome given these conditions.
For example, after hearing the first few words of
a sentence, you will develop an expectation of which
word is likely to come next. The state of the neurons
upon generating this expectation is the minus phase.
The experience of the actual word that comes next es-