Information Technology Reference
In-Depth Information
engineering, statistical research, and so on. RL has particularly rich roots in the
psychology of animal learning, from which it takes its name. A number of
impressive applications of RL have also been developed. RL has attracted much
research in the past decade. Its incremental nature and adaptive capabilities make
it suitable for use in various domains, such as automatic control, mobile robotics
and multi-agent system. Particularly with the breakthrough of the mathematical
basis of reinforcement learning, the application of RL increasingly conducted, as
one of the hot spots of the current research in the field of machine learning.
The history of reinforcement learning had two main threads, both long and
rich, which were pursued independently before intertwining into modern
reinforcement learning. One thread concerned learning by trial and error and
started in the psychology of animal learning. This thread run through some of the
earliest work in artificial intelligence and led to the revival of reinforcement
learning in the early 1980s. The other thread concerned the problem of optimal
control and its solution using value functions and dynamic programming. For the
most part, this thread did not involve learning. Although the two threads were
largely independent, the exceptions revolve around a third, less distinct thread
concerning temporal-difference methods. All three threads came together in the
late 1980s to produce the modern field of reinforcement learning.
In early artificial intelligence, before it was distinct from other branches of
engineering, several researchers began to explore trial-and-error learning as an
engineering principle. The earliest computational investigations of trial-and-error
learning were perhaps by Minsky and Farley and Clark, both in 1954. In his Ph.D.
dissertation, Minsky discussed computational models of reinforcement learning
and described his construction of an analog machine; composed of components
he called SNARCs (Stochastic Neural-Analog Reinforcement Calculators).
Farley and Clark described another neural-network learning machine designed to
learn by trial-and-error. In the 1960s one finds the terms “reinforcement” and
“reinforcement learning” being widely used in the engineering literature for the
first time. Particularly influential was Minsky's paper “Steps Toward Artificial
Intelligence”(Minsky,1961), which discussed several issues relevant to
reinforcment learning, including what he called the credit-assignment problem:
how do you distribute credit for success among the many decisions that may have
been involved in producing it? In 1969 Minsky got Turing Award in computer
due to above contribution.
In 1994 and 1995 the interests of Farley and Clark shifted from trial-and-error
learning to generalization and pattern recognition, that is, from reinforcement
