Information Technology Reference
In-Depth Information
Table 11.2
Results on the TD2004 dataset
Algorithm
NDCG@1
NDCG@3
NDCG@10
P@1
P@3
P@10
MAP
Regression
0.360
0.335
0.303
0.360
0.333
0.249
0.208
RankSVM
0.413
0.347
0.307
0.413
0.347
0.252
0.224
RankBoost
0.507
0.430
0.350
0.507
0.427
0.275
0.261
FRank
0.493
0.388
0.333
0.493
0.378
0.262
0.239
ListNet
0.360
0.357
0.317
0.360
0.360
0.256
0.223
AdaRank
0.413
0.376
0.328
0.413
0.369
0.249
0.219
SVM
map
0.293
0.304
0.291
0.293
0.302
0.247
0.205
Table 11.3
Results on the NP2003 dataset
Algorithm
NDCG@1
NDCG@3
NDCG@10
P@1
P@3
P@10
MAP
Regression
0.447
0.614
0.665
0.447
0.220
0.081
0.564
RankSVM
0.580
0.765
0.800
0.580
0.271
0.092
0.696
RankBoost
0.600
0.764
0.807
0.600
0.269
0.094
0.707
FRank
0.540
0.726
0.776
0.540
0.253
0.090
0.664
ListNet
0.567
0.758
0.801
0.567
0.267
0.092
0.690
AdaRank
0.580
0.729
0.764
0.580
0.251
0.086
0.678
SVM
map
0.560
0.767
0.798
0.560
0.269
0.089
0.687
Table 11.4
Results on the NP2004 dataset
Algorithm
NDCG@1
NDCG@3
NDCG@10
P@1
P@3
P@10
MAP
Regression
0.373
0.555
0.653
0.373
0.200
0.082
0.514
RankSVM
0.507
0.750
0.806
0.507
0.262
0.093
0.659
RankBoost
0.427
0.627
0.691
0.427
0.231
0.088
0.564
FRank
0.480
0.643
0.729
0.480
0.236
0.093
0.601
ListNet
0.533
0.759
0.812
0.533
0.267
0.094
0.672
AdaRank
0.480
0.698
0.749
0.480
0.244
0.088
0.622
SVM
map
0.520
0.749
0.808
0.520
0.267
0.096
0.662
It is clear that the larger
S
i
(M)
is, the better the
i
th algorithm performs. For ease
of reference, we refer to this measure as the
winning number
. Table
11.8
shows the
winning number for all the algorithms under investigation. From this table, we have
the following observations.
1. In terms of NDCG@1, P@1 and P@3, the listwise ranking algorithms perform
the best, followed by the pairwise ranking algorithms, while the pointwise rank-
ing algorithm performs the worst. Among the three listwise ranking algorithms,
ListNet is better than AdaRank and SVM
map
. The three pairwise ranking al-
