Information Technology Reference
In-Depth Information
5.4 Study other applications of modeling ranking as a sequence of classification
tasks, rather than defining the essential loss and deriving the upper bounds.
5.5 It has been observed in [
9
] that sometimes optimizing one measure on the train-
ing data might not lead to the best ranking performance on the test data in terms
of the same measure. Explain this phenomenon and design an experiment to
validate your hypothesis.
References
1. Agarwal, S., Graepel, T., Herbrich, R., Har-Peled, S., Roth, D.: Generalization bounds for the
area under the roc curve. Journal of Machine Learning
6
, 393-425 (2005)
2. Chakrabarti, S., Khanna, R., Sawant, U., Bhattacharyya, C.: Structured learning for non-
smooth ranking losses. In: Proceedings of the 14th ACM SIGKDD International Conference
on Knowledge Discovery and Data Mining (KDD 2008), pp. 88-96 (2008)
3. Chen, W., Liu, T.-Y., Lan, Y., Ma, Z., Li, H.: Ranking measures and loss functions in learning
to rank. In: Advances in Neural Information Processing Systems 22 (NIPS 2009), pp. 315-323
(2010)
4. Cossock, D., Zhang, T.: Subset ranking using regression. In: Proceedings of the 19th Annual
Conference on Learning Theory (COLT 2006), pp. 605-619 (2006)
5. He, Y., Liu, T.-Y.: Tendency correlation analysis for direct optimization methods. Information
Retrieval
13
(6), 657-688 (2010)
6. Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the 8th
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD
2002), pp. 133-142 (2002)
7. Li, P., Burges, C., Wu, Q.: McRank: learning to rank using multiple classification and gradient
boosting. In: Advances in Neural Information Processing Systems 20 (NIPS 2007), pp. 845-
852 (2008)
8. Qin, T., Liu, T.-Y., Li, H.: A general approximation framework for direct optimization of
information retrieval measures. Information Retrieval
13
(4), 375-397 (2009)
9. Talyor, M., Guiver, J., et al.: Softrank: optimising non-smooth rank metrics. In: Proceedings
of the 1st International Conference on Web Search and Web Data Mining (WSDM 2008),
pp. 77-86 (2008)
10. Xu, J., Liu, T.-Y., Lu, M., Li, H., Ma, W.-Y.: Directly optimizing IR evaluation measures in
learning to rank. In: Proceedings of the 31st Annual International ACM SIGIR Conference on
Research and Development in Information Retrieval (SIGIR 2008), pp. 107-114 (2008)
11. Yue, Y., Finley, T., Radlinski, F., Joachims, T.: A support vector method for optimizing aver-
age precision. In: Proceedings of the 30th Annual International ACM SIGIR Conference on
Research and Development in Information Retrieval (SIGIR 2007), pp. 271-278 (2007)
