Changing Not Just Analyzing: Control Theory and Reinforcement Learning - Realtime Data Mining - page 14

Chapter 3
Changing Not Just Analyzing: Control
Theory and Reinforcement Learning
Abstract We give a short introduction to reinforcement learning. This includes
basic concepts like Markov decision processes, policies, state-value and action-
value factions, and the Bellman equation. We discuss solution methods like policy
and value iteration methods, online methods like temporal-difference learning, and
state fundamental convergence results.
It turns out that RL addresses the problems from Chap. 2 . This shows that, in
principle, RL is a suitable instrument for solving all of these problems.
We have described how a good recommendation engine should learn step by step
by interaction with its environment. It is precisely this task that reinforcement
learning (RL), one of the most fascinating disciplines of machine learning,
addresses. RL is used among other things to control autonomous systems such as
robots and also for self-learning games like backgammon or chess. And as we will
see later, despite all problems, RL turns out to be an excellent framework for
recommendation engines.
In this chapter, we present a brief introduction to reinforcement learning before
in the subsequent chapter we consider its application to REs. For a detailed
introduction, we refer you to the standard work “Reinforcement Learning - An
Introduction” by Richard Sutton and Andrew Barton [SB98], from which some of
the figures in this chapter have been taken. Especially, following [SB98] for reasons
of a unified treatment, we will unify the model-based approach, the dynamic
programming, as well as the model-free approach, the actual reinforcement learn-
ing, under the term “reinforcement learning.”