Best and fairest: an empirical analysis of retrieval system bias

Colin Wilkie, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

16 Citations (Scopus)

Abstract

In this paper, we explore the bias of term weighting schemes used by retrieval models. Here, we consider bias as the extent to which a retrieval model unduly favours certain documents over others because of characteristics within and about the document. We set out to find the least biased retrieval model/weighting. This is largely motivated by the recent proposal of a new suite of retrieval models based on the Divergence From Independence (DFI) framework. The claim is that such models provide the fairest term weighting because they do not make assumptions about the term distribution (unlike most other retrieval models). In this paper, we empirically examine whether fairness is linked to performance and answer the question; is fairer better?
Original languageEnglish
Title of host publicationAdvances in Information Retrieval
Subtitle of host publication36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings
Place of PublicationNew York, NY, USA
PublisherSpringer International Publishing AG
Pages13-25
Number of pages13
ISBN (Print)978-3-319-06027-9
DOIs
Publication statusPublished - 2014

Publication series

NameLecture Notes in Computer Science
PublisherSpringer International Publishing
Volume8416
ISSN (Print)0302-9743

Keywords

  • information retrieval
  • retrieval models
  • term weighting

Cite this

Wilkie, C., & Azzopardi, L. (2014). Best and fairest: an empirical analysis of retrieval system bias. In Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings (pp. 13-25). (Lecture Notes in Computer Science; Vol. 8416). New York, NY, USA: Springer International Publishing AG. https://doi.org/10.1007/978-3-319-06028-6_2