Information Technology Reference
In-Depth Information
a)
Press View again and select INFLECTIONS . Click
on the D event button (for the /D/ inflectional phoneme)
on the left side of the window, and also the - button for
the blank pattern.
By comparing these patterns with the weights in the
last slot, you can see that the unit codes for either the
D inflection (equivalent to d ) or the blank. Therefore,
it is very clear that this unit plays an important role in
producing the regular past tense inflection. Presumably,
other units that compete with these units get more acti-
vated by the irregular words, and thus are able to sup-
press the regular inflection. It is the sensitive dynamics
of this kind of competition as the network settles that
contributes to a relatively strong U-shaped overregular-
ization curve, as we will see.
Overregularization in Leabra
1.0
0.8
0.6
0.4
OR
Responses
0.2
0.0
0
25000
50000
75000
Number of Words
b)
Overregularization in Bp
1.0
0.8
Analyze the production of the progressive -inginflec-
tion, using using the same technique we just used for the
past tense inflection (the -inginflectional semantics start
at the third unit from the left in the last row).
0.6
, !
0.4
OR
Responses
0.2
Question 10.9 (a) Do the most active units code for
the appropriate inflectional phonological pattern? (b)
Describe the steps you took to reach this answer.
0.0
0
25000
50000
75000
Number of Words
Although it is interesting to see something about
how the network has learned this task, the most
relevant empirical data is in the time-course of its
overregularizations during training. We automati-
cally scored the phonological output from the net-
work for each trial as it learned, looking for over-
regularizations of irregular past tense words, in addi-
tion to a number of other patterns of output that are
not discussed here (see the files pt_all.css and
pt_words_all_full.list for details).
Figure 10.20 shows the plot of overregularizations
for both the Leabra network we have just been explor-
ing and a standard feedforward backpropagation net-
work (Bp) run on the same task. Note that, following
convention, overregularization is plotted as 1 minus the
proportion of overregularization errors, which gives the
characteristic U-shape that is evident in the graph. The
plots also show the proportion of phonologically valid
responses that the network made for past-tense verbs,
which is important for evaluating when overregulariza-
tions occur relative to any period of early correct re-
sponding.
Figure 10.20: Plot of overregularizations and total responses
as the network learns for (a) Leabra and (b) backpropagation
(Bp). In Leabra there is a clear initial period where overreg-
ularizations are absent, but the network is producing valid re-
sponses for up to 50 percent of the past-tense verbs, but this
is not present in the Bp network. Then, overregularization in
Leabra (plotted as 1 minus the number of overregularizations)
begins and continues at a low sporadic rate for an extended
period, whereas overregularization in Bp is resolved relatively
quickly. Thus, the Leabra network, but not Bp, provides a rea-
sonable fit to the developmental data.
There are two critical U-shaped curve features that
are evident in the comparison between the Leabra and
Bp networks. First, the Leabra network achieves a sub-
stantial level (around 50%) of responding prior to the
onset of overregularization. Thus, the network demon-
strates an early-correct period of irregular verb produc-
tion, where irregular verbs are being produced with-
out overregularization errors. This is the critical aspect
of the empirical data that previous models have failed
Search WWH ::




Custom Search