How Engines Learn to Generate Recommendations: Adaptive Learning Algorithms - Realtime Data Mining - page 60

8

7

6

PVs (manual)
Revenue (manual)
PVs P-Version
Revenue P-Version

5

Prediction
rate

4

3

2

1

0

0 5000 10000 15000

Number of steps

Fig. 5.1 Prediction rates against the number of transaction steps

12,000 steps, the P-Version attains a rate of almost 7.5 %, which in the course of the
day increases to no more than about 8.5 %. Taking preceding days into account
yields an increase of only about 10 %.

■

5.2 Conditional Approach

Next, we proceed a step further than the unconditional approach and consider the
solution of ( 3.5 ) as a control problem, i.e., using the conditional probabilities p ss 0 .
To this end, we need to simplify the determination of the transition probabilities
p ss 0 . We make the following assumption introduced by Thess:

Assumption 5.2 (Conditional probability property): For each state transition
from s to s 0 under the action a , for which s 0 is not the state s a associated with
the action a , the transition probability p ss 0 is considered as proportional to the
unconditional probability p ss 0 , i.e., p ss 0 p ss 0 .

It follows that (in addition to the direct determination of p ss a )

p ss 0 ¼ cs s 0 ð p ss 0 , ; 6¼ s a ð 5 : 3 Þ

where c ( s , a ) is a constant depending only on s and a . We will call the approach
based on Assumption 5.2 the DP-Version .
In other words, if a recommendation a is delivered but not accepted, and instead
of the recommended product s a , the user clicks on another product s 0 , we assume
that

the user selected this independently of the current recommendation a .