Database Reference
In-Depth Information
Chapter 5
How Engines Learn to Generate
Recommendations: Adaptive Learning
Algorithms
Abstract This chapter is mainly devoted to the question of estimating transition
probabilities taking into account the effect of recommendations. It turned out that
this is an extremely complex problem. The central result is a simple empirical
assumption that allows reducing the complexity of the estimation in a way that is
computationally suitable to most practical problems. The discussion of this
approach gives a deeper insight into essential principles of realtime recommenda-
tion engines. Based on this assumption, we propose methods to estimate the
transition probabilities and provide some first experimental results. Although the
results look promising, more advanced techniques are highly desirable. Such
techniques like hierarchical and factorization methods are presented in the follow-
ing chapters.
In Chap.
4
, we have gathered all the ingredients to apply the reinforcement learning
approaches described in Chap.
3
to recommendation engines. Except for one thing,
we still do not have the transition probabilities
p
ss
0
! These are really problematic,
because we would have to save not only all the product transitions
s ! s
0
but also
those for all recommendations
a
that are generated. For large web shops, in
particular, this can result in huge numbers of rules, of the order of magnitude of
all transactions and thus containing thousands of millions of rules. Not only would
this be technically difficult, it would also be extremely unstable, because most of
those rules would have hardly any statistical basis.
Therefore, we must make some empirical assumptions in order to achieve
plausible simplifications. The simplest approach is the classical one: we simply
ignore the recommendations
a
. We therefore work only with the transition proba-
bilities
p
ss
0
, i.e., without considering the actions
a
. We refer to
p
ss
0
as
unconditional
transition probabilities
, in contrast to
conditional transition probabilities p
ss
0
.
