Language - Computational Explorations in Cognitive Neuroscience - page 350

priming-like overregularization phenomenon that is ab-
sent in the feedforward backpropagation network (Bp).
Again, Hebbian learning does not increase the effect,
and here we find that with a larger Hebbian level (.005),
overregularization is actually decreased.
The null or detrimental effects of Hebbian learning in
this large-scale model are somewhat inconsistent with
smaller-scale models of the dynamic competition and
priming account of overregularization as explored by
O'Reilly and Hoeffner (in preparation). Nevertheless,
it is clear that the major effect here is in the activa-
tion dynamics that facilitate the competition between
the irregular and regular mappings, and although Heb-
bian learning does not facilitate the effects, it does fa-
cilitate our ability to interpret the network's weights as
we did above. The impairment in overregularizations
with larger amounts of Hebbian learning (.005) can be
attributed to its tendency to differentiate (cluster sepa-
rately, see chapter 4) the irregular and regular mappings
such that overregularization becomes less likely.

gitudinal data available for only a handful of cases, and
recording starting well after the onset of language pro-
duction. Thus, it is difficult to determine if a U-shaped
curve is truly universal, or is just seen in a subset of
individuals (Hoeffner, 1997).
Second, it is difficult to achieve a valid mapping be-
tween network learning rates and human learning rates.
The network is completely focused on acquiring the
semantics to phonology mapping on each trial, and it
learns exceedingly rapidly compared with children. For
example, the network achieved well over 90 percent
valid responding to irregular pasts after only 25,000 tri-
als of training.
It is difficult to know how to map the model'stri-
als onto a child's learning experience. For example, if
we assume that the child is exposed to at most 8 min-
utes of speech per day (at a rate of one word per second)
that contributes to their learning of semantics to phonol-
ogy, the equivalent time period to achieve 90 percent
valid responding for the child would amount to only 52
days! The entire time span of the U-shaped curve in the
network, and complete mastery of the corpus (roughly
75,000 trials), would be just 156 days, rather than the
years observed empirically. Although one could ap-
ply some kind of generic scaling to the network'sper-
formance, it is likely that a major part of the differ-
ence is due to the complexity and subtlety of the larger
language acquisition task that the child is performing.
Thus, it is probably premature to expect these relatively
simple models to provide a detailed chronological pic-
ture.
Finally, one advantage of the above model relative to
others in the literature is that it uses a completely static
training environment. The network's gradual learning
of this environment results in a gradual expansion of
its productive vocabulary, which is a nice contrast to
the external imposition of this expansion by the re-
searcher. Nevertheless, a child's semantic representa-
tions are gradually developing during this same time
period, and this may place important constraints on
the learning process that are not reflected in our static
model.
Thus, probably the best interpretation of this model is
that it demonstrates that a network with a static environ-
ment can exhibit a U-shaped curve, and that it does so

Go to the PDP++Root window. To continue on to
the next simulation, close this project first by selecting
.projects/Remove/Project_0 . Or, if you wish to
stop now, quit by selecting Object/Quit .

10.5.3 Summary and Discussion

This exploration shows that two important mechanis-
tic principles of activation flow incorporated into the
Leabra model, interactivity and inhibitory competition,
produce a reasonable account of the developmental U-
shaped overregularization curve in terms of competi-
tion between the irregular and regular mappings and
priming-like effects from the regular mapping that lead
to overregularizations (O'Reilly & Hoeffner, in prepa-
ration). The Leabra network exhibits both a more sub-
stantial early correct responding period, and a much
more extended overregularization period with many
more total overregularizations, as compared to the kind
of feedforward backpropagation network used in most
previous models.
However, despite these encouraging findings, a more
detailed and rigorous evaluation of the model's fitto
the human data is complicated by several factors. First,
the human data are woefully underspecified, with lon-