Introduction - Learning to Rank for Information Retrieval - page 19

Fig. 1.6 Learning-to-rank framework

m (i) is the number of documents associated with query q i ), and the corresponding
relevance judgments. 19 Then a specific learning algorithm is employed to learn the
ranking model (i.e., the way of combining the features), such that the output of the
ranking model can predict the ground truth label in the training set 20 as accurately as
possible, in terms of a loss function. In the test phase, when a new query comes in,
the model learned in the training phase is applied to sort the documents and return
the corresponding ranked list to the user as the response to her/his query.
Many learning-to-rank algorithms can fit into the above framework. In order to
better understand them, we perform a categorization on these algorithms. In partic-
ular, we group the algorithms, according to the four pillars of machine learning, into
three approaches: the pointwise approach, the pairwise approach, and the listwise
approach. Different approaches model the process of learning to rank in different
ways. That is, they may define different input and output spaces, use different hy-
potheses, and employ different loss functions.

19 Please distinguish the judgment for evaluation and the judgment for constructing the training set,
although the process may be very similar.
20 Hereafter, when we mention the ground-truth labels in the remainder of the topic, we will mainly
refer to the ground-truth labels in the training set, although we assume every document will have
its intrinsic label, no matter whether it is judged or not.