Database Reference
In-Depth Information
User enters the
shop via A
A
User goes to
product B
B
RE recommends
product C
User goes to
product D
D
C
C*
D
RE recommends
product E
User goes to
product E
E*
E
E
RE recommends
product F
F
F*
Fig. 4.1 Example of reinforcement learning for product recommendations in a web shop
the existing transactions are mostly clicks, whereas, on the other hand, placements
in the shopping basket (SB) and purchases are far more infrequent. However,
maximizing sales is the primary goal of REs. Let us summarize these two
problems again:
1. High numbers of products, a majority of which have minimal transaction
history.
2. The vast majority of transactions are clicks; only a fraction are placements in
the shopping basket and purchases.
We know, however, from the theory and practice of RL that high transaction
numbers are necessary in order to achieve convergence. The above problems
therefore appear to be killer arguments against the direct use of RL for REs.
There already exist first approaches for using reinforcement learning for recom-
mendation engines [GR04, RSP05, SHB05, TGK07, Mah10]. However, most of
them are not able to overcome the complexity problems.
Therefore, additional empirical assumptions are made and justified below, which
reduce the complexity of using RL for REs.
