Language - Computational Explorations in Cognitive Neuroscience - page 359

processing involve ways of manipulating these kinds of
hierarchical tree structures to produce and comprehend
sentences.
For the traditional tree-structured approach to sen-
tence comprehension to work, one must build up the
appropriate tree structure by identifying the correspond-
ing role of each of the words in a sentence. Thus, a
major aspect of this view of syntax involves using var-
ious cues (e.g., word order) to bind words to roles. For
the example shown in figure 10.26, one needs to know
that “the boy” is the subject of the sentence, and not
the direct object. In this case, the fact that “boy” is the
first noun in the sentence is a very reliable cue (in En-
glish) that it is the subject. However, things can become
much more complicated when the sentence structure
gets more complex (e.g., with passive constructions, de-
pendent clauses and the like), and when the words have
multiple grammatical categories (e.g., “run” can be ei-
ther a verb or a noun).
In short, the traditional view of sentence processing
becomes a combinatorial search problem of large pro-
portions, with the correct tree structure for a given sen-
tence being one out of a huge number of possible such
trees. In contrast, a neural network approach to sen-
tence comprehension does not require explicit category
labels, or even any kind of explicit representation of the
tree structure of a sentence. All that is required is a suf-
ficiently rich distributed representation of the meaning
of the sentence, constructed as a result of reading the
constituent words.
By way of analogy, consider the difference between
the traditional and neural network approaches to object
recognition (see chapter 8). The traditional approach
posited that an internal object-centered 3-D structural
model (e.g., as might be produced in a computer aided
design (CAD) program) was constructed from the per-
ceptual input. With such a model, recognizing the ob-
ject should be simple. However, because the detailed
3-D structure is obscured by the 2-D projection re-
ceived by the eyes, constructing the 3-D model is a
massive underconstrained combinatorial search prob-
lem that turned out to be unworkable.
In contrast, the neural network approach only tries to
come up with an internal distributed representation that
disambiguates different objects, while being roughly in-

S

NP VP

(subject)

V NP

(direct object)

Art N

The boy chases the cats

Figure 10.26: Tree diagram of the phrase structure of a sim-
ple English sentence. NP = noun phrase, VP = verb phrase,
and S is represents the entire sentence.

pletely ignored the issue of syntax. In this section we
explore a model developed originally by St. John and
McClelland (1990) that combines syntactic and seman-
tic information in the service of sentence-level process-
ing. To begin our discussion of this model, we first in-
troduce some of the main issues in syntactic processing.
The term syntax generally refers to the structural reg-
ularities of sentences, or in other words, the grammar
of sentences. We have already seen that neural network
models can capture linguistic regularities at the level
of the pronunciation and inflection of individual words.
Critically, these models demonstrate that principles of
neural network learning and processing can be used to
understand the kinds of regularities that have tradition-
ally been ascribed to the operation of explicit grammati-
cal rules. Thus, neural network models should provide a
similar kind of demonstration in the context of sentence
level processing.
The traditional view of syntactic processing involves
hierarchically structured tree diagrams of the phrase
structure of a sentence, like that shown in figure 10.26.
This diagram represents the fact that every well-formed
English sentence has a noun phrase (NP) and a verb
phrase (VP). These phrases can then be further de-
composed into subcomponents until the actual words
in the sentence are represented. This same information
can be expressed in terms of rewrite rules that specify
how to rewrite or decompose something into its con-
stituent parts. For example, the top-level structure of
figure 10.26 can be written as S ! NP VP , meaning
that a sentence is composed of a noun phrase and a verb
phrase. Traditional computational theories of syntactic