TY - GEN
T1 - Retrievability and retrieval bias
T2 - 37th European Conference on Information Retrieval Research, ECIR 2015
AU - Wilkie, Colin
AU - Azzopardi, Leif
PY - 2015
Y1 - 2015
N2 - The disposition of a retrieval system to favour certain documents over others can be quantified using retrievability. Typically, the Gini Coefficient has been used to quantify the level of bias a system imposes across the collection with a single value. However, numerous inequality measures have been proposed that may provide different insights into retrievability bias. In this paper, we examine 8 inequality measures, and see the changes in the estimation of bias on 3 standard retrieval models across their respective parameter spaces. We find that most of the measures agree with each other, and that the parameter setting that minimise the inequality according to each measure is similar. This work suggests that the standard inequality measure, the Gini Coefficient, provides similar information regarding the bias. However, we find that Palma index and 20:20 Ratio show the greatest differences and may be useful to provide a different perspective when ranking systems according to bias.
AB - The disposition of a retrieval system to favour certain documents over others can be quantified using retrievability. Typically, the Gini Coefficient has been used to quantify the level of bias a system imposes across the collection with a single value. However, numerous inequality measures have been proposed that may provide different insights into retrievability bias. In this paper, we examine 8 inequality measures, and see the changes in the estimation of bias on 3 standard retrieval models across their respective parameter spaces. We find that most of the measures agree with each other, and that the parameter setting that minimise the inequality according to each measure is similar. This work suggests that the standard inequality measure, the Gini Coefficient, provides similar information regarding the bias. However, we find that Palma index and 20:20 Ratio show the greatest differences and may be useful to provide a different perspective when ranking systems according to bias.
UR - http://www.scopus.com/inward/record.url?scp=84925433064&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-16354-3_22
DO - 10.1007/978-3-319-16354-3_22
M3 - Conference contribution book
AN - SCOPUS:84925433064
VL - 9022
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 209
EP - 214
BT - Advances in Information Retrieval - 37th European Conference on IR Research, ECIR 2015, Proceedings
PB - Springer-Verlag
CY - Switzerland
Y2 - 29 March 2015 through 2 April 2015
ER -