Efficiently estimating retrievability bias

Colin Wilkie, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

8 Citations (Scopus)

Abstract

Retrievability is the measure of how easily a document can be retrieved using a particular retrieval system. The extent to which a retrieval system favours certain documents over others (as expressed by their retrievability scores) determines the level of bias the system imposes on a collection. Recently it has been shown that it is possible to tune a retrieval system by minimising the retrievability bias. However, to perform such a retrievability analysis often requires posing millions upon millions of queries. In this paper, we examine how many queries are needed to obtain a reliable and useful approximation of the retrievability bias imposed by the system, and an estimate of the individual retrievability of documents in the collection. We find that a reliable estimate of retrievability bias can be obtained, in some cases, with 90% less queries than are typically used while estimating document retrievability can be done with up to 60% less queries.
Original languageEnglish
Title of host publicationAdvances in Information Retrieval
Subtitle of host publication36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings
Place of PublicationNew York, NY, USA
PublisherSpringer International Publishing AG
Pages720-726
Number of pages7
ISBN (Print)978-3-319-06027-9
DOIs
Publication statusPublished - 2014

Publication series

NameLecture Notes in Computer Science
PublisherSpringer International Publishing
Volume8416
ISSN (Print)0302-9743

Keywords

  • information retrieval
  • system bias
  • query formulation
  • document retrievability

Cite this

Wilkie, C., & Azzopardi, L. (2014). Efficiently estimating retrievability bias. In Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings (pp. 720-726). (Lecture Notes in Computer Science; Vol. 8416). Springer International Publishing AG. https://doi.org/10.1007/978-3-319-06028-6_82