How Engines Learn to Generate Recommendations: Adaptive Learning Algorithms - Realtime Data Mining - page 65

5.2.2 Special Cases

We investigate and interpret below important special cases of ( 5.6 ). For this we
firstly write out in full ( 5.6 ) with ( 5.5 ):

X

1 p ss a
1 p ss a

q π s ðÞ¼p ss a r ss a þ

; p ss 0 r ss 0 : ð 5 : 8 Þ

s 0 6¼s a

The following special cases arise:
1. p ss a ¼ p ss a

(no effectiveness of the recommendation):

ðÞ¼p ss a r ss a þ X
s 0 6¼s a

p ss 0 r ss 0 ¼ X
s 0

q π s p ss 0 r ss 0 ¼ q 0 ; ðÞ ,

and all recommendations a lead to the same q 0 action value
2. p ss a ¼ 0 (no acceptance of the recommendation):

X

1
1 p ss a

q π s ðÞ¼ ; p ss 0 r ss 0 :

s 0 6¼s a

The interpretation is that the action value of the recommendation
a corresponds to the weighted unconditional action value. The conditional action
value disappears. The reward for the recommendation plays no role at all, since
there never is a transition to the product s a .
3. p ss a ¼ 1 (total acceptance of the recommendation):

q π s ðÞ¼r ss a , ;

which means that the recommendation a always obtains its full reward. The
unconditional action value disappears.
4. p ss a ¼ 0 (no acceptance of the “recommendation” in the control group):

X

q π s ðÞ¼p ss a r ss a þ 1 p ss a ; p ss 0 r ss 0 :

s 0 6¼s a

The interpretation is the action value corresponds to the conditional action
value of recommendation a plus the probability of nonacceptance of the recom-
mendation times the unconditional action value for all the other states s 0 6¼ s a .
5. p ss a ¼ 1 (total acceptance of the “recommendation” in the control group):

1

k X
s 0 6¼s a

q π s ðÞ¼p ss a r ss a þ 1 p ss a ; r ss 0 ,