Retrievability and retrieval bias: a comparison of inequality measures

Colin Wilkie, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

5 Citations (Scopus)


The disposition of a retrieval system to favour certain documents over others can be quantified using retrievability. Typically, the Gini Coefficient has been used to quantify the level of bias a system imposes across the collection with a single value. However, numerous inequality measures have been proposed that may provide different insights into retrievability bias. In this paper, we examine 8 inequality measures, and see the changes in the estimation of bias on 3 standard retrieval models across their respective parameter spaces. We find that most of the measures agree with each other, and that the parameter setting that minimise the inequality according to each measure is similar. This work suggests that the standard inequality measure, the Gini Coefficient, provides similar information regarding the bias. However, we find that Palma index and 20:20 Ratio show the greatest differences and may be useful to provide a different perspective when ranking systems according to bias.

Original languageEnglish
Title of host publicationAdvances in Information Retrieval - 37th European Conference on IR Research, ECIR 2015, Proceedings
Place of PublicationSwitzerland
Number of pages6
ISBN (Electronic)9783319163536
Publication statusPublished - 2015
Event37th European Conference on Information Retrieval Research, ECIR 2015 - Vienna, Austria
Duration: 29 Mar 20152 Apr 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference37th European Conference on Information Retrieval Research, ECIR 2015


Dive into the research topics of 'Retrievability and retrieval bias: a comparison of inequality measures'. Together they form a unique fingerprint.

Cite this