Robotics Reference
In-Depth Information
goal for the contestants, of course, is to be the first to program a com-
puter whose responses are indistinguishable from human conversation, a
goal for which Loebner has pledged a grand prize of $100,000 and a gold
medal. The winner of the annual contest is the best entry, irrespective
of how good or bad is its conversational ability relative to those of the
human judges at the competition.
The competition rules specify that a panel of judges should each in-
teract with a number of computer terminals, at least one of which is
linked to a computer program (a contestant) and at least one of which is
linked to a human “confederate”. Each entity should respond in such a
manner as to attempt to convince the judges that the entity is a human
being. All of the judges conduct conversations via all of the terminals
and then rank the terminals from most-humanlike to least-humanlike.
The computer program that achieves the highest average ranking is the
winner. In some years the competition has allowed each program entry
to specify only a single topic of conversation to be discussed, while in
other years, including 1997 when my team won the prize,
10
the judges
have been able to converse on whatever subjects they wished.
The Loebner Prize has become a regular benchmarking experiment
in NLP. By comparing the quality of the winning program entry each
year it is possible to monitor progress in the field of conversational com-
puting. Earlier in this chapter are the transcripts of two conversations by
ELIZA from the mid-1960s. Take a look at the following extract from
a transcript, one of four conversations between the judges at the 2004
Loebner Prize competition and the winning entry, the ALICE program
developed by Richard Wallace, and decide for yourself whether the state
of the art in 2004 represented a significant advance, or indeed any ad-
vance, on Weizenbaum's work of four decades earlier. (Incidentally, this
was the third time that ALICE had won the Loebner Prize.)
ALICE:
Hello, my name is Alice and I am the hu-
man.
Judge 4:
Hi, I'm Roy. Are you a Yankees fan?
ALICE:
Hi there Judge 4. Good gossip: you are Roy.
Am I a Yankees fan? Maybe
Judge 4:
Maybe. You are obviously not from NY.
10
Details of all the winners since the competition started in 1990, and all the competition tran-
scripts, are available at
http://www.loebner.net/
.
