Database Reference
In-Depth Information
Table 5.9 Cumulated rewards for P-, DP-, and P-DP-Versions for simulations over virtual
sessions
γ
P-Version
DP-Version
P-DP-Version (
n
min
¼
50)
0.0
443,005.74
536,764.94
540,197.18
0.5
435,934.75
584,838.17
585,926.50
1.0
450,942.64
608,496.66
601,278.30
In reality the connection between conditional and unconditional probabilities is,
of course, more complex and constitutes a qualitative mixture of all three cases
a)-c). Fortunately, the case a), where the conditional probability is higher than the
unconditional one, dominates as we have also seen in Sect.
5.4.1
. This explains why
the P-Version in most practical applications works very well.
■
Example 5.5
Now we return to the data set of Example 5.2 and run a virtual
simulation with 100,000 sessions. We use 3 recommendations and the estimation of
the transition probabilities
p
ss
0
was performed by Algorithm 5.3.
We now again compare the P-Version with the DP-Version for different discount
rates. Additionally, we test the combined P-DP-Version with the threshold value
n
min
¼
50. In all versions, the ADP Algorithm 3.3 was applied, in which the full
calculation of the state-value function in line 9 was provided for each 1,000th session.
Table
5.9
shows the results.
From the results we see that the DP-Version clearly outperforms P-Version. Also
the chain optimization works well, for the DP-Version better than for the P-Version.
In contrast, the results of the P-DP-Version do not indicate a clear improvement
compared to the DP-Version.
■
5.5 Summary
This chapter was mainly devoted to the question of estimating transition probabilities
taking into account the effect of recommendations. It turned out that this is an
extremely complex problem. The central result was a simple empirical assumption
that allows reducing the complexity of the estimation in a way that is suitable to most
practical problems. The discussion of this approach gave a deeper insight into
essential principles of recommendation engines. Based on this assumption we pro-
posed methods to estimate the transition probabilities and provided some first
experimental results. Although the results look promising, more advanced techniques
are highly desirable. This will be the central topic of the next chapters.
