Language - Computational Explorations in Cognitive Neuroscience - page 340

the processing of any given input. As we know from
the nature of the mapping problem itself, lots of subtle
countervailing forces must be balanced out to determine
how to pronounce a given word. Finally, the fact that we
can easily interpret some units' weights is due to the use
of Hebbian learning, which causes the weights to reflect
the probabilities of unit co-occurrence.

Do PickUnit again and select HID_I , and then
click on the left-most Ortho_Code unit that has a strong
(yellow) weight value.
We will explore a somewhat more complex example
of invariant coding now, where an Ortho_Code unit
represents two different letters, “o” and “w,” across dif-
ferent locations.

PickUnit OC_OW .
Although the connections are somewhat more faint
(because they are distributed more widely), you should
be able to see that this unit receives most strongly from
the letter “o” in all 3 of the ending locations, and the
letter “w” in the last two locations. Now, let's trace the
strongest connection from this unit to the hidden layer.

Poke around some more at the network's weights,
and document another relatively clear example of
how the representations across the Ortho_Code and
Hidden layers make sense in terms of the input/output
mapping being performed.

, ! , !

Question 10.7 (a) Specify what Ortho_Code units
you have chosen, what letters those Ortho_Code
units encode, how the hidden unit(s) combine the
Ortho_Code units together, and what phonemes the
hidden unit(s) produce on the output. (b) Relate your
analysis to the need for both spatial invariance and con-
junctive encoding.

PickUnit HID_OW .
You should see that the hidden unit projects most
strongly to a single feature in the vowel slot — as we
will see, this feature is shared by the /u/ /W/ vowel
phonemes.

, !

Verify this by doing View on VOWELS and selecting
u and W , while comparing with the HID_OW weights.
This mapping from 'o' and 'w' to the /W/ vowel is
obviously relevant for the regular mapping from words
like how ! /hW/, and the activation of the /u/ phoneme
makes sense if this unit is also activated by other
Ortho_Code units that should activate that pronunci-
ation (you can find evidence for this in the weights, but
the connections are not as strong and are thus harder to
see). This last set of units demonstrates that more com-
plex conjunctions of input letters are represented in the
network as well, which is also reminiscent of layer V2
units in the object recognition model, and of the wick-
elfeatures of the SM89 model and the hand-tuned con-
junctive units in the PMSP model.
There are several important lessons from this exercise
in tracing the weights. First, the network seems to learn
the right kinds of representations to allow for good gen-
eralization. These representations are similar to those of
the V2 layer of the object recognition model in that they
combine spatial invariance with conjunctive feature en-
coding. Second, although we are able to obtain insight
by looking at some of the representations, not all are so
easily interpretable. Further, once the network'scom-
plex activation dynamics are figured into the picture, it
is even more difficult to figure out what is happening in

Nonword Pronunciation

We next test the network's ability to generalize by pro-
nouncing nonwords that exploit the regularities in the
spelling to sound mapping. A number of nonword sets
exist in the literature — we use three sets that PMSP
used to test their model. The first set of nonwords
is comprised of two lists, the first derived from regu-
lar words, the second from exception words (Glushko,
1979). The second set was constructed to determine
if nonwords that are homophones for actual words are
pronounced better than those which are not, so the set
is also comprised of two lists, a control list and a homo-
phone list (McCann & Besner, 1987). The third set of
nonwords were derived from the regular and exception
probe word lists that we used to test the network earlier
(Taraban & McClelland, 1987).

Do PickTest and select GLUSHKO . Then
StepTest (don't forget to click back on act in the net-
work).
Looking at the Trial_1_TextLog , we can see
that network correctly pronounced the nonword “beed”
by producing /bbbEddd/.

, !