Recommendations as a Game: Reinforcement Learning for Recommendation Engines - Realtime Data Mining - page 49

Algorithm 4.2: (continued)

1: procedure UPDATE_P_MULTI( j p, a , l ,

α j )

: ¼ F j p

j p

2: ⊳ conversion into composite

probabilities

jþ 1 p

: ¼ UPDATE_P_SINGLE( j p , l ,

3: α l ) ⊳ update of composite

probabilities

jþ 1 p

: ¼ F 1 jþ 1 p

4: ⊳ conversion into single

probabilities

5: return j +1 p
6: end procedure

In practice, the function F together with its inverse F 1 turns out to be very
helpful tools for converting back and forth between the transition probabilities of
single and multiple recommendations.
We conclude with a fi na l remark. So far, we have only considered one fixed
multiple recommen d ation a ¼ a 1 , ð , a k Þ . However, in reality, different multiple

...

recommendations a i ¼ a i 1 , ð Þ are issued subsequentially, and even the

number of th ei r single r e commendations k may vary. Different multiple recom-
mendations

, a i k

...

a i and a j often share similar single recommendations, i.e.,

a s 0 , leading to the update of the same single probabilities p a s 0 a i l ¼ a j m ¼ :

ss 0 .This
raises the questions of the consistency and stability of Algorithm 4.2 when applied
to all recommendations.
Let V be the space of all multiple recommendations p a i

ss 0

and their

“recommended” states s 0 :

8
<

9
=
; :

ss 0 s 0 ∈S a i , i∈ Ρ s j 0 p a i ss 0 1, X

s 0 ∈

p a i p a i V

: ¼

ss 0 1

:

S a i

Here, we consider all multiple recommendations a i over the power set P n
where n is the number of all products and P s is the corresponding index set. Further,
let W be the space of all single recommendations p a s 0

ss 0 :

n o

ss 0 s 0 ∈S AðÞ j 0 p a s 0

p a s 0 W

: ¼

ss 0 1

:

We rewrite ( 4. 6 ) by explicitly mentioning that it applies to a particular multiple
recommendation a i :

ss 0 s 0 , p a i

p a i ,

ss 0 s 0 , F a i

: ¼ p ¼ p a s 0 p a i ¼ F a i p a i p a i : ¼ p ¼ : ¼ F :

Next, we extend F a i to F a G such that it works on the whole space W b y simply
ignoring all single probabilities not belonging to the recommendations of a i .By
p G ¼ p a i i ¼ F a G p ðÞ

i ¼ :

F G p ð , p G ∈ V p G ∈ W

, ,