Best and fairest: an empirical analysis of retrieval system bias

Colin Wilkie; Leif Azzopardi

doi:10.1007/978-3-319-06028-6_2

Best and fairest: an empirical analysis of retrieval system bias

Colin Wilkie, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

30 Citations (Scopus)

Abstract

In this paper, we explore the bias of term weighting schemes used by retrieval models. Here, we consider bias as the extent to which a retrieval model unduly favours certain documents over others because of characteristics within and about the document. We set out to find the least biased retrieval model/weighting. This is largely motivated by the recent proposal of a new suite of retrieval models based on the Divergence From Independence (DFI) framework. The claim is that such models provide the fairest term weighting because they do not make assumptions about the term distribution (unlike most other retrieval models). In this paper, we empirically examine whether fairness is linked to performance and answer the question; is fairer better?

Original language	English
Title of host publication	Advances in Information Retrieval
Subtitle of host publication	36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings
Place of Publication	New York, NY, USA
Publisher	Springer International Publishing AG
Pages	13-25
Number of pages	13
ISBN (Print)	978-3-319-06027-9
DOIs	https://doi.org/10.1007/978-3-319-06028-6_2
Publication status	Published - 2014

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer International Publishing
Volume	8416
ISSN (Print)	0302-9743

Keywords

information retrieval
retrieval models
term weighting

Access to Document

10.1007/978-3-319-06028-6_2

Leif Azzopardi
- leif.azzopardistrath.acuk
- Computer And Information Sciences - Reader
- Strathclyde ischool
Person: Academic

Cite this

Wilkie, C., & Azzopardi, L. (2014). Best and fairest: an empirical analysis of retrieval system bias. In Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings (pp. 13-25). (Lecture Notes in Computer Science; Vol. 8416). Springer International Publishing AG. https://doi.org/10.1007/978-3-319-06028-6_2

@inproceedings{3806a646c6d148d9802514213780077b,

title = "Best and fairest: an empirical analysis of retrieval system bias",

abstract = "In this paper, we explore the bias of term weighting schemes used by retrieval models. Here, we consider bias as the extent to which a retrieval model unduly favours certain documents over others because of characteristics within and about the document. We set out to find the least biased retrieval model/weighting. This is largely motivated by the recent proposal of a new suite of retrieval models based on the Divergence From Independence (DFI) framework. The claim is that such models provide the fairest term weighting because they do not make assumptions about the term distribution (unlike most other retrieval models). In this paper, we empirically examine whether fairness is linked to performance and answer the question; is fairer better?",

keywords = "information retrieval, retrieval models, term weighting",

author = "Colin Wilkie and Leif Azzopardi",

year = "2014",

doi = "10.1007/978-3-319-06028-6_2",

language = "English",

isbn = "978-3-319-06027-9",

series = "Lecture Notes in Computer Science",

publisher = "Springer International Publishing AG",

pages = "13--25",

booktitle = "Advances in Information Retrieval",

address = "Switzerland",

}

Wilkie, C & Azzopardi, L 2014, Best and fairest: an empirical analysis of retrieval system bias. in Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings. Lecture Notes in Computer Science, vol. 8416, Springer International Publishing AG, New York, NY, USA, pp. 13-25. https://doi.org/10.1007/978-3-319-06028-6_2

Best and fairest: an empirical analysis of retrieval system bias. / Wilkie, Colin; Azzopardi, Leif.
Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings. New York, NY, USA: Springer International Publishing AG, 2014. p. 13-25 (Lecture Notes in Computer Science; Vol. 8416).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

TY - GEN

T1 - Best and fairest

T2 - an empirical analysis of retrieval system bias

AU - Wilkie, Colin

AU - Azzopardi, Leif

PY - 2014

Y1 - 2014

N2 - In this paper, we explore the bias of term weighting schemes used by retrieval models. Here, we consider bias as the extent to which a retrieval model unduly favours certain documents over others because of characteristics within and about the document. We set out to find the least biased retrieval model/weighting. This is largely motivated by the recent proposal of a new suite of retrieval models based on the Divergence From Independence (DFI) framework. The claim is that such models provide the fairest term weighting because they do not make assumptions about the term distribution (unlike most other retrieval models). In this paper, we empirically examine whether fairness is linked to performance and answer the question; is fairer better?

AB - In this paper, we explore the bias of term weighting schemes used by retrieval models. Here, we consider bias as the extent to which a retrieval model unduly favours certain documents over others because of characteristics within and about the document. We set out to find the least biased retrieval model/weighting. This is largely motivated by the recent proposal of a new suite of retrieval models based on the Divergence From Independence (DFI) framework. The claim is that such models provide the fairest term weighting because they do not make assumptions about the term distribution (unlike most other retrieval models). In this paper, we empirically examine whether fairness is linked to performance and answer the question; is fairer better?

KW - information retrieval

KW - retrieval models

KW - term weighting

U2 - 10.1007/978-3-319-06028-6_2

DO - 10.1007/978-3-319-06028-6_2

M3 - Conference contribution book

SN - 978-3-319-06027-9

T3 - Lecture Notes in Computer Science

SP - 13

EP - 25

BT - Advances in Information Retrieval

PB - Springer International Publishing AG

CY - New York, NY, USA

ER -

Wilkie C, Azzopardi L. Best and fairest: an empirical analysis of retrieval system bias. In Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. Proceedings. New York, NY, USA: Springer International Publishing AG. 2014. p. 13-25. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-06028-6_2

Best and fairest: an empirical analysis of retrieval system bias

Abstract

Publication series

Keywords

Access to Document

Fingerprint

Profiles

Leif Azzopardi

Cite this