How Engines Learn to Generate Recommendations: Adaptive Learning Algorithms - Realtime Data Mining - page 83

3,00

2,50

2,00

rsC
rsC_ctrl

1,50

1,00

0,50

0,00

0 10 20 30 40 50 60

Update step min( n, m )

Fig. 5.4 Average probability ratios rs C and rs C _ ctrl for recommendation and control groups

We summarize that first tests indicate the correctness of Assumption 5.2.
However, more advanced instruments, like factorizations presented in Chaps. 8 ,
9 , and 10 , are required to increase its effectiveness.

5.4.2 Extension of the Simulation

We first estimate the model of the environment from the transaction data. This
model later will enable us to create an arbitrary number of virtual sessions.
Therefore we subsequently process the t ra nsaction data described in Sect. 4.4
and calculate the transition probabilities p ss 0 using the Algorithm 5.2. For our
environment model we also need the transition rewards r ss 0 that we estimate by
( 3.8 ) in conjunction with Assumption 4.2. To get results of the form of Table 4.2
which includes baskets and orders, we follow a more granular approach, and for all
transitions s ! s 0 we estimate the probabilities that product s 0 will be afterward
added to the basket as well as the average number of finally ordered units. This
enables us to simulate additionally the numbers of baskets and orders as well as the
revenue instead the click number only.
In order to generate virtual sessions, we need to know when a session terminates.
Here the absorbing state s A comes to the aid. A s soon as we reach the absorbing
state according to the transition probability p ss A , we terminate the session. There
only remains the question which products to select at the beginning of the sessions.
For this we introduce the generating state s G which can be viewed as counterpart to