When is query performance prediction effective?

Claudia Hauff, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

4 Citations (Scopus)

Abstract

The utility of Query Performance Prediction (QPP) methods is commonly evaluated by reporting correlation coefficients to denote how well the methods perform at predicting the retrieval performance of a set of queries. However, a quintessential question remains unexplored: how strong does the correlation need to be in order to realize an increase in retrieval performance? In this work, we address this question in the context of Selective Query Expansion (SQE) and perform a large-scale experiment. The results show that to consistently and predictably improve retrieval effectiveness in the ideal SQE setting, a Kendall's Tau correlation of tau>=0.5 is required, a threshold which most existing query performance prediction methods fail to reach.
Original languageEnglish
Title of host publicationSIGIR '09 Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval
Place of PublicationNew York, NY, USA
Pages829-830
Number of pages2
DOIs
Publication statusPublished - 19 Jul 2009
Externally publishedYes

Keywords

  • evaluation
  • query performance prediction

Cite this

Hauff, C., & Azzopardi, L. (2009). When is query performance prediction effective? In SIGIR '09 Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 829-830). https://doi.org/10.1145/1571941.1572150