Machine learning and natural language processing - The Computing Universe: A Journey through a Revolution - page 291

Fig. 14.9. The Kinect tracking system
needs to cope with a huge variety of
body poses, shapes, and sizes.

Speech and language processing

The idea of using computers to process and understand human language
dates back to Turing and Shannon, two of the earliest pioneers of computing.
Research into speech and language processing now covers a wide range of sub-
disciplines ranging from computational linguistics and natural language pro-
cessing in computer science, to speech recognition and speech synthesis in
electrical engineering. In the 1950s, building on earlier work on language gram-
mars by Shannon, Noam Chomsky ( B.14.4 ) introduced the idea of a “context-free
grammar” - a mathematically precise formalism to describe how phrases in a
natural language are built from smaller “blocks.” Chomsky's grammar was able
to capture the way in which clauses nest inside other clauses and how adjec-
tives and adverbs are associated to nouns and verbs. Chomsky's most famous
proposal is that knowledge of the formal grammar of a language explains the
ability of a human to produce and interpret an infinite number of sentences
from a limited set of rules and words. A second research direction in this early
period was the development of probabilistic algorithms for speech and lan-
guage processing. In 1952, researchers at Bell Labs built the first Automatic
Speech Recognizer (ASR) system. This was a statistical system that could recog-
nize the first ten digits with 97 percent to 99 percent accuracy - provided the
speaker was male, spoke with a 350 millisecond delay between words, and the
machine had been tuned to the speaker's voice profile. Otherwise the accuracy
of the system fell to about 60 percent.
During the 1960s and 1970s, speech and language processing followed both
these directions of research - the formal symbolic approach of Chomsky and
the statistical approach of ASR systems. The symbolic approach was typically
picked up by the emerging artificial intelligence (AI) community while the sta-
tistical approach was mainly followed by electrical engineers and statisticians.
Two examples illustrate the different approaches of these two paradigms. As an
example application from the symbolic research community, we can take Terry
Winograd's SHRDLU system, written in 1972. This system was able to simulate
the behavior of a robot interacting with a world of toy blocks. The program was
able to accept sophisticated text commands in natural language such as “Find
a block which is taller than the one you are holding and put it into the box.”

B.14.4. Noam Chomsky is a linguist,
philosopher, cognitive scientist, and
an outspoken defender of democ-
racy and human rights. Since 1955
he has been a professor at MIT.
Chomsky is a prolific author who
has published more than a hundred
topics. His idea that children have
an innate body of linguistic knowl-
edge - the Universal Grammar - has
been immensely influential in
linguistics. His topics about politics,
economy, and society have often
been controversial.