Information Technology Reference
In-Depth Information
The third case of continuous trajectories is one that
we will not focus on very much. Here, the relevant in-
formation is in the detailed temporal evolution of what
might be best described as a continuous system, rather
than just in the order of a discrete set of events as char-
acterized by the sequential case. This kind of continu-
ous information is probably quite important for motor
control and some perceptual tasks (e.g., perception of
motion).
We do not focus on continuous trajectory learning
because it is complex and does not fit well within our
existing framework. Mechanistically, representing a
continuous trajectory depends critically on the detailed
temporal response characteristics of neurons. From
a biological perspective, many parameters other than
weights are likely to be critical, with the self-regulating
channels described in chapter 2 providing just one ex-
ample of the considerable complexities in temporal re-
sponse that biological neurons can exhibit. Thus, adapt-
ing these parameters in a useful way would require
very different kinds of mechanisms than we have ex-
plored previously. Furthermore, the available compu-
tational mechanisms for learning continuous trajecto-
ries (e.g., Williams & Zipser, 1989; Pearlmutter, 1989;
Williams & Peng, 1990) are far from biologically plau-
sible, and they do not tend to work very well on any-
thing but relatively simple tasks either.
Nevertheless, it may be possible to model some con-
tinuous trajectory phenomena by “digitizing” a trajec-
tory into a sequence of events by taking a set of snap-
shots at regular intervals, and then use the mechanisms
developed for sequential tasks. We do not mean to
imply that sequential learning is mutually exclusive
with temporally delayed learning — indeed, we actually
think there may be an intimate relationship between the
two, as discussed further in chapters 9 and 11.
Learning in sequential or temporally delayed tasks is
generally much more difficult than learning direct in-
put/output mappings. Adding intervening time steps
between input-output contingencies can be a lot like
adding extra hidden layers to a network. Indeed, there
is a way of learning over time with backpropagation
where each additional time step is equivalent to adding
a new hidden layer between the input and output layers
(Rumelhart et al., 1986a). Thus, given the advantages of
the model learning biases (specifically Hebbian learn-
ing and inhibitory competition) for learning in deep net-
works, one might expect that they will also be useful for
learning temporally extended tasks. The example we
explore below indicates that this is likely the case. We
will follow this up with more interesting and complex,
cognitively relevant tasks in the second part of the topic.
We first explore the use of context representations to
deal with learning sequential tasks, and then move on to
the use of reinforcement learning to handle temporally
delayed learning.
6.6
Context Representations and Sequential
Learning
The central problem in learning sequential tasks is de-
veloping useful context representations that capture the
information from previous events that is needed to pro-
duce an appropriate output (or interpretation) at some
later point in time. In the simplest case, called the
Markovian case, the only prior context necessary to
predict the next step in the sequence is contained in the
immediately preceding time step. This case is particu-
larly convenient because it avoids the storage problems
(and consequent decisions about what information to
throw away) that arise when the context must include
information from many prior time steps.
In a non-Markovian task, one must expand the
context representation (i.e., prior state representation)
to contain all the necessary contingencies from prior
states. A trivial example of this is one where the
context representation literally contains a copy of all
prior states. Obviously, such a strategy is impossible
with limited capacity memory systems, so some kind
of adaptive, strategic memory system must be used to
maintain just the right context information. Although
the neural network mechanisms we explore here are not
particularly powerful in this respect, chapter 9 shows
how these mechanisms can be augmented with a more
intelligent control mechanism to successfully handle
non-Markovian tasks.
Two closely related types of neural network mod-
els that incorporate a Markovian-style context repre-
sentation were developed by Jordan (1986) and El-
man (1990), and are often called Jordan and Elman
Search WWH ::




Custom Search