Query variation performance prediction for systematic reviews

Harrisen Scells, Leif Azzopardi, Guido Zuccon, Bevan Koopman

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

6 Citations (Scopus)
21 Downloads (Pure)

Abstract

When conducting systematic reviews, medical researchers heavily deliberate over the final query to pose to the information retrieval system. Given the possible query variations that they could construct, selecting the best performing query is difficult. This motivates a new type of query performance prediction (QPP) task where the challenge is to estimate the performance of a set of query variations given a particular topic. Query variations are the reductions, expansions and modifications of a given seed query under the hypothesis that there exists some variations (either generated from permutations or hand crafted) which will improve retrieval effectiveness over the original query. We use the CLEF 2017 TAR Collection, to evaluate sixteen pre and post retrieval predictors for the task of Query Variation Performance Prediction (QVPP). Our findings show the IDF based QPPs exhibits the strongest correlations with performance. However, when using QPPs to select the best query, little improvement over the original query can be obtained, despite the fact that there are query variations which perform significantly better. Our findings highlight the difficulty in identifying effective queries within the context of this new task, and motivates further research to develop more accurate methods to help systematic review researchers in the query selection process.

Original languageEnglish
Title of host publication41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018
Place of PublicationNew York, NY.
Pages1089-1092
Number of pages4
ISBN (Electronic)9781450356572
DOIs
Publication statusPublished - 27 Jun 2018
Event41st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018 - Ann Arbor, United States
Duration: 8 Jul 201812 Jul 2018

Conference

Conference41st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018
CountryUnited States
CityAnn Arbor
Period8/07/1812/07/18

Keywords

  • Information retrieval
  • systematic reviews
  • query formulation
  • information retrieval
  • HCI

Fingerprint Dive into the research topics of 'Query variation performance prediction for systematic reviews'. Together they form a unique fingerprint.

  • Cite this

    Scells, H., Azzopardi, L., Zuccon, G., & Koopman, B. (2018). Query variation performance prediction for systematic reviews. In 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018 (pp. 1089-1092). https://doi.org/10.1145/3209978.3210078