Database Reference
In-Depth Information
t
s
,
s
0
t
u
,
s
0
:¼
X
s
,
k
þ
X
v
s
,
s
0
s
,
k
v
s
0
,
s
v
u
,
s
0
u
,
k
v
s
0
,
u
s
0
,
k
,
s
0
e
a
u
,
ss
0
∈
S
:
ð
9
:
9
Þ
k¼
1
k¼
1
Intuitively, this may be put as
a
MC
a
CF
e
a
u
,
ss
0
¼ e
u
,
ss
0
þe
u
,
ss
0
:
ð
9
:
10
Þ
Here, the first term
MC
corresponds to a Markov transition, as considered in
collaborative filtering, which has been presented in the previous chapter by means
of Example 8.1. Thus, this factorization unifies both approaches in a simple
manner. Since the parameters of both modes are learned jointly, the approach is by
no means trivial.
Even though the approach appears simple, it brings along a fair amount of difficul-
ties. First, the question of approximation error arises, since, as a matter of fact, the
approach brings about a great deal of simplification with arguable plausibility.
Furthermore, the factorized “probabilities” are, of course, no longer stochastic and
thus not probabilities. Granted, the authors of [RFST10] perform an additional trans-
formation by a sigmoid function such that
0
a
u
,
ss
0
1,
8s
,
s
0
∈
S
holds. But even then the row sum condition (
3.2
) is violated. With regard to the
goals of [RFST10], this is not an issue, since the recommendations are derived
directly from the probabilities and, therefore, only an ordering of these needs to be
ensured. For our purposes of RL, we may not ignore this condition, but must again
demand:
X
a
u
,
ss
0
¼
1
8u
∈
U
∧8s
∈
S
:
s
0
Hence, we need to incorporate these conditions in the solution procedure. Sadly
enough, this doesn't make the computation of the factorization, which is compli-
cated enough in itself, any easier. Apart from that, another difficulty, though not
related to the factorization itself, comes into play: allowing the transition probabil-
ities to depend on the session violates Assumption 4.1 of the Markov property. In
transition probabilities as functions of the course of the session and yet satisfies the
(generalized) Markov property.
