Language - Computational Explorations in Cognitive Neuroscience - page 335

ulars can be thought of as extreme examples of context
dependency where the pronunciation is dependent on
the configuration of the entire word. Thus, there is a
continuum between regularity and irregularity. A neu-
ral network with appropriately trained weights can deal
with this continuum very naturally by taking all the ap-
propriate contingencies into account in making its re-
sponse. In contrast, the traditional rule-based account of
the direct spelling-to-sound pathway requires an elabo-
rate and improbable collection of “rules” to deal with
the properties of this mapping

tion of all wickelfeatures (including an initial and fi-
nal “blank” represented by ) contained in the word.
For example, _think_ was represented by _th thi ,
hin , ink ,and nk_ . Although the conjunctive nature
of these representations was useful for capturing the in-
terdependence of pronunciation on other letters as dis-
cussed above, it did not allow for combinatorial rep-
resentations of letters (e.g., the representation of the
b in bank was completely different from that of the b
in blank ). This lack of combinatoriality would obvi-
ously impair the network's ability to generalize to novel
words properly.
The model developed by Plaut et al. (1996) (the
PMSP model) corrected the problems with the SM89
representations by having single units represent letters
and phonemes regardless of their surrounding context
(i.e., combinatorial representations). Specifically, they
divided the word into three parts, the onset , vowel ,and
coda , with one set of letter/phoneme representations for
each of these parts. For example, the word think
would be represented by the t and h units in the on-
set (plus one other unit as described below), the i unit
in the vowel, and the n and k , units in the coda. This
makes the mapping between letter and pronunciation
very systematic because the same representations are
reused over and over again regardless of their surround-
ing context.
However, one could imagine that the extreme combi-
natoriality in PMSP would lead to confusion over the
exact nature of the word, because order information
within a given part is not actually encoded in the input
(i.e., think and htikn are represented identically).
This isn't much of a problem in English (as evident in
this example) because there aren't many different words
that differ only in letter order within one of the three
parts (onset, vowel, coda). The PMSP model also added
some conjunctive representations that have very specific
pronunciation consequences, such as th , sh ,and ch ,
because here the order is quite significant for pronun-
ciation (this conjunctive unit is the extra one active in
think referred to above). A similar scheme was de-
veloped for the output phonology, combining combina-
torial and conjunctive elements.
In short, the PMSP model used hand-tuned represen-
tations that significantly simplified and regularized the

(e.g., Coltheart et al.,

1993).
Given the existence of regularities like those de-
scribed above, it makes sense that proficient English
speakers pronounce a novel nonword like bint just like
mint and not like pint . Given the complex nature of the
regularities, this systematic behavior must be appropri-
ately sensitive to conjunctions of letters in some cases
(e.g., so that bine is pronounced like mine while bint
is pronounced like mint ), but also appropriately insen-
sitive to these conjunctions so that the initial b can be
pronounced the same way independent of the other let-
ters. Thus, the correct generalization performance for
nonwords depends critically on representations that are
both conjunctive (depending on letter conjunctions like
-int and -ine ), and combinatorial (allowing arbitrary
combinations, like the initial b being combined with -int
and -ine ). One goal of neural network models of read-
ing is to show how such appropriate representations can
develop through learning to pronounce the known En-
glish words.
Seidenberg and McClelland (1989) developed a neu-
ral network model of reading that learned to per-
form the orthography to phonology mapping for nearly
3,000 monosylabic English words (hereafter the SM89
model). However, due to an unfortunate choice of in-
put/output representations, this model did not general-
ize to pronounceable nonwords very well, failing to pro-
duce the same kinds of systematic pronunciations that
people produce for the same word inputs.
The representations in the SM89 model were called
wickelfeatures , due to their conjunctive nature inspired
by the ideas of Wickelgren (1979). Each wickelfeature
represented a conjunction of three letters or phonemes.
An entire word input was represented by the activa-