The Pairwise Approach - Learning to Rank for Information Retrieval - page 60

Fig. 3.8 Training multiple

rankers

the model trained from categories k and l is denoted by f k,l , then the final ranking
results can be attained by using the weighted BordaCount aggregation. 4

f(x) = α k,l f k,l (x). (3.20)

k,l

Here the combination coefficient α k,l can be pre-specified or learned from a sep-
arate validation set. The experimental results in [ 26 ] show that by considering more
information about the judgment, the ranking performance can be significantly im-
proved over Ranking SVM. Note that the technology used in MHR actually can be
extended to any other pairwise ranking algorithms.

3.3.2 Magnitude-Preserving Ranking

In [ 14 ], Cortes et al. also attempt to tackle the first problem with the pairwise
approach. In particular, they propose keeping the magnitude of the labeled pref-
erences, and accordingly use the so-called magnitude-preserving loss (MP loss),
hinge magnitude-preserving loss (HMP loss), and SVM regression loss (SVR loss)
for learning to rank. These three loss functions can effectively penalize a pairwise
misranking by the magnitude of predicted preference or the β th power of that mag-
nitude. That is, considering the property of the power function x β , if the loss on a
pair is large, it will be highly penalized if β is set to be large.

4 Note that there are many algorithms for rank aggregation proposed in the literature, such as Bor-
daCount [ 2 , 5 , 16 ], median rank aggregation [ 17 ], genetic algorithm [ 4 ], fuzzy logic-based rank
aggregation [ 1 ], and Markov chain-based rank aggregation [ 16 ]. Although BordaCount is used
in [ 26 ] as an example, it by no means dictates that other methods cannot be used for the same
purpose.