Robotics Reference
In-Depth Information
Backgammon knowledge, Tesauro added Neurogammon's hand-designed
evaluation features to the TD networks. The result was that TD-Gam-
mon greatly surpassed Neurogammon and all other previous computer
programs. This was demonstrated in numerous test games played by TD-
Gammon against several world-class human grandmasters, including Bill
Robertie and Paul Magriel, both noted authors and highly respected for-
mer World Champions. A later version of TD-Gammon (version 2.1)
even came very close to defeating Bill Robertie in a 40-game test ses-
sion. Robertie actually trailed the entire session, and only in the very last
game was he able to pull ahead for an extremely narrow one-point vic-
tory. Robertie thought that in at least a few cases during the session, the
program had come up with some genuinely novel strategies that actually
improved on the way top humans usually play.
TD-Gammon's success, and the perception that it had reached the
level of human world class players, had an enormous influence on the de-
velopment of future Backgammon programs. In the subsequent decade
two commercially available programs were launched, both employing
neural networks and both playing at or above the level of TD-Gammon.
First came Jellyfish, written in Norway by Fredrik Dahl, which was
launched in 1997, and then Snowie, written in Switzerland by Olivier
Egger. These programs are widely regarded as being on a par with or
perhaps even stronger than the world's best human players.
47
Poker—Imperfect and Misleading Information, and Chance
As with most card games, Poker is a game of imperfect information be-
cause some or all of the opponents' cards are hidden. Poker also has addi-
tional levels of complexity that do not pertain to any of the other games
we have discussed thus far—risk management (the various possible bet-
ting strategies and their consequences), opponent modelling (identifying
patterns in each opponent's strategy and exploiting this knowledge), de-
ception (bluffing and varying one's style of play), and dealing with unre-
liable information (taking into account your opponent's deceptive plays).
These extra dimensions, much like decision-making problems in the real
world, make the programming of Poker an extremely challenging task,
yet the past few years have seen the development of a Poker program that
performs at the level of very strong human players.
47
As of January 2005 there had not yet been a showdown in a match of a statistically significant
length between any of these leading programs and the reigning human World Champion.
