Information Technology Reference
In-Depth Information
16.2.3 The Listwise Approach
For the listwise approach, let
be the output space, whose elements are permuta-
tions of
m
documents, denoted as
π
y
. Then
(
x
,π
y
)
can be regarded as a random
variable sampled from the space
Y
m
X
×
Y
according to an unknown probability dis-
tribution
P
.
Suppose the listwise loss function is
L(f
;
x
,π
y
)
. Then the expected risk can be
represented as
R(f )
=
L(f
;
x
,π
y
)P (d
x
,dπ
y
).
(16.13)
m
X
×
Y
Intuitively, the expected risk means the loss that a ranking model
f
would make
for all the
m
documents associated with a random query
q
. Since it is almost im-
possible to compute the expected risk, in practice, the empirical risk on the training
set is used as an estimate of the expected risk. In particular, given the training data
{
(
x
(i)
,π
(i
y
)
n
}
i
=
1
,the
empirical risk
can be defined as follows:
n
L
f
;
x
(i)
,π
(i
y
.
1
n
R(f )
=
(16.14)
i
=
1
16.3 Two-Layer Ranking Framework
As pointed out in [
9
] and [
4
], the aforementioned two frameworks have their limi-
tations in analyzing the ranking algorithms for information retrieval. The document
ranking framework ignores the existence of queries, while the subset ranking frame-
work ignores the sampling of documents. In contract, it is possible to sample both
more queries and more documents to label in real information retrieval scenarios.
Therefore, one should consider both queries and documents as random variables
and investigate the theoretical properties when the numbers of both queries and
documents approach infinity. This is exactly the motivation of the two-layer ranking
framework.
16.3.1 The Pointwise Approach
Let
be the query space. Each query
q
is assumed to be a random variable sam-
pled from the query space with an unknown probability distribution
P
Q
Q
. Given this
query,
(x
j
,y
j
)
is assumed to be a random variable sampled according to probability
distribution
D
q
(which is dependent on query
q
).
