Information Technology Reference
In-Depth Information
1.3.3.1 The Pointwise Approach
The
input space
of the pointwise approach contains a feature vector of each single
document.
The
output space
contains the relevance degree of each single document. Differ-
ent kinds of judgments can be converted to ground truth labels in terms of relevance
degree:
•
If the judgment is directly given as relevance degree
l
j
, the ground truth label for
document
x
j
is defined as
y
j
=
l
j
.
•
If the judgment is given as pairwise preference
l
u,v
, one can get the ground truth
label by counting the frequency of a document beating other documents.
•
If the judgment is given as the total order
π
l
, one can get the ground truth label
by using a mapping function. For example, the position of the document in
π
l
can
be used as the ground truth.
The
hypothesis
space contains functions that take the feature vector of a docu-
ment as input and predict the relevance degree of the document. We usually call
such a function
f
the scoring function. Based on the scoring function, one can sort
all the documents and produce the final ranked list.
The
loss function
examines the accurate prediction of the ground truth label for
each single document. In different pointwise ranking algorithms, ranking is mod-
eled as regression, classification, and ordinal regression, respectively (see Chap. 2).
Therefore the corresponding regression loss, classification loss, and ordinal regres-
sion loss are used as the loss functions.
Example algorithms belonging to the pointwise approach include [
13
-
15
,
19
-
21
,
26
,
28
,
44
,
47
,
55
,
68
,
78
]. We will introduce some of them in Chap. 2.
Note that the pointwise approach does not consider the inter-dependency between
documents, and thus the position of a document in the final ranked list is invisible
to its loss function. Furthermore, the approach does not make use of the fact that
some documents are actually associated with the same query. Considering that most
evaluation measures for information retrieval are query level and position based, the
pointwise approach has its limitations.
1.3.3.2 The Pairwise Approach
The
input space
of the pairwise approach contains pairs of documents, both repre-
sented by feature vectors.
The
output space
contains the pairwise preference (which takes values from
{+
) between each pair of documents. Different kinds of judgments can be
converted to ground truth labels in terms of pairwise preferences:
1
,
−
1
}
•
If the judgment is given as relevance degree
l
j
, then the pairwise preference for
(
x
u
,x
v
) can be defined as
y
u,v
=
·
I
{
l
u
l
v
}
−
2
1.
•
If the judgment is given directly as pairwise preference, then it is straightforward
to set
y
u,v
=
l
u,v
.
