Retrievability and retrieval bias: a comparison of inequality measures

Colin Wilkie, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

5 Citations (Scopus)

Abstract

The disposition of a retrieval system to favour certain documents over others can be quantified using retrievability. Typically, the Gini Coefficient has been used to quantify the level of bias a system imposes across the collection with a single value. However, numerous inequality measures have been proposed that may provide different insights into retrievability bias. In this paper, we examine 8 inequality measures, and see the changes in the estimation of bias on 3 standard retrieval models across their respective parameter spaces. We find that most of the measures agree with each other, and that the parameter setting that minimise the inequality according to each measure is similar. This work suggests that the standard inequality measure, the Gini Coefficient, provides similar information regarding the bias. However, we find that Palma index and 20:20 Ratio show the greatest differences and may be useful to provide a different perspective when ranking systems according to bias.

Original languageEnglish
Title of host publicationAdvances in Information Retrieval - 37th European Conference on IR Research, ECIR 2015, Proceedings
Place of PublicationSwitzerland
PublisherSpringer-Verlag
Pages209-214
Number of pages6
Volume9022
ISBN (Electronic)9783319163536
DOIs
Publication statusPublished - 2015
Event37th European Conference on Information Retrieval Research, ECIR 2015 - Vienna, Austria
Duration: 29 Mar 20152 Apr 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9022
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference37th European Conference on Information Retrieval Research, ECIR 2015
CountryAustria
CityVienna
Period29/03/152/04/15

Cite this