Back to the roots

mean-variance analysis of relevance estimations

Guido Zuccon, Leif Azzopardi, Keith van Rijsbergen

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

Abstract

Recently, mean-variance analysis has been proposed as a novel paradigm to model document ranking in Information Retrieval. The main merit of this approach is that it diversifies the ranking of retrieved documents. In its original formulation, the strategy considers both the mean of relevance estimates of retrieved documents and their variance. However, when this strategy has been empirically instantiated, the concepts of mean and variance are discarded in favour of a point-wise estimation of relevance (to replace the mean) and of a parameter to be tuned or, alternatively, a quantity dependent upon the document length (to replace the variance). In this paper we revisit this ranking strategy by going back to its roots: mean and variance. For each retrieved document, we infer a relevance distribution from a series of point-wise relevance estimations provided by a number of different systems. This is used to compute the mean and the variance of document relevance estimates. On the TREC Clueweb collection, we show that this approach improves the retrieval performances. This development could lead to new strategies to address the fusion of relevance estimates provided by different systems.
Original languageEnglish
Title of host publicationECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval
Place of PublicationBerlin, Heidelberg
PublisherSpringer-Verlag
Pages716-720
Number of pages5
ISBN (Print)978-3-642-20160-8
Publication statusPublished - 18 Apr 2011
Externally publishedYes

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Verlag
Volume6611

Fingerprint

analysis of variance
ranking
information retrieval
paradigm
performance

Keywords

  • information retrieval
  • mean variance analysis

Cite this

Zuccon, G., Azzopardi, L., & van Rijsbergen, K. (2011). Back to the roots: mean-variance analysis of relevance estimations. In ECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval (pp. 716-720). (Lecture Notes in Computer Science; Vol. 6611). Berlin, Heidelberg: Springer-Verlag.
Zuccon, Guido ; Azzopardi, Leif ; van Rijsbergen, Keith. / Back to the roots : mean-variance analysis of relevance estimations. ECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval. Berlin, Heidelberg : Springer-Verlag, 2011. pp. 716-720 (Lecture Notes in Computer Science).
@inproceedings{824b8f021bd1437ca62ecd28c91bca1e,
title = "Back to the roots: mean-variance analysis of relevance estimations",
abstract = "Recently, mean-variance analysis has been proposed as a novel paradigm to model document ranking in Information Retrieval. The main merit of this approach is that it diversifies the ranking of retrieved documents. In its original formulation, the strategy considers both the mean of relevance estimates of retrieved documents and their variance. However, when this strategy has been empirically instantiated, the concepts of mean and variance are discarded in favour of a point-wise estimation of relevance (to replace the mean) and of a parameter to be tuned or, alternatively, a quantity dependent upon the document length (to replace the variance). In this paper we revisit this ranking strategy by going back to its roots: mean and variance. For each retrieved document, we infer a relevance distribution from a series of point-wise relevance estimations provided by a number of different systems. This is used to compute the mean and the variance of document relevance estimates. On the TREC Clueweb collection, we show that this approach improves the retrieval performances. This development could lead to new strategies to address the fusion of relevance estimates provided by different systems.",
keywords = "information retrieval, mean variance analysis",
author = "Guido Zuccon and Leif Azzopardi and {van Rijsbergen}, Keith",
year = "2011",
month = "4",
day = "18",
language = "English",
isbn = "978-3-642-20160-8",
series = "Lecture Notes in Computer Science",
publisher = "Springer-Verlag",
pages = "716--720",
booktitle = "ECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval",

}

Zuccon, G, Azzopardi, L & van Rijsbergen, K 2011, Back to the roots: mean-variance analysis of relevance estimations. in ECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval. Lecture Notes in Computer Science, vol. 6611, Springer-Verlag, Berlin, Heidelberg, pp. 716-720.

Back to the roots : mean-variance analysis of relevance estimations. / Zuccon, Guido; Azzopardi, Leif; van Rijsbergen, Keith.

ECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval. Berlin, Heidelberg : Springer-Verlag, 2011. p. 716-720 (Lecture Notes in Computer Science; Vol. 6611).

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - Back to the roots

T2 - mean-variance analysis of relevance estimations

AU - Zuccon, Guido

AU - Azzopardi, Leif

AU - van Rijsbergen, Keith

PY - 2011/4/18

Y1 - 2011/4/18

N2 - Recently, mean-variance analysis has been proposed as a novel paradigm to model document ranking in Information Retrieval. The main merit of this approach is that it diversifies the ranking of retrieved documents. In its original formulation, the strategy considers both the mean of relevance estimates of retrieved documents and their variance. However, when this strategy has been empirically instantiated, the concepts of mean and variance are discarded in favour of a point-wise estimation of relevance (to replace the mean) and of a parameter to be tuned or, alternatively, a quantity dependent upon the document length (to replace the variance). In this paper we revisit this ranking strategy by going back to its roots: mean and variance. For each retrieved document, we infer a relevance distribution from a series of point-wise relevance estimations provided by a number of different systems. This is used to compute the mean and the variance of document relevance estimates. On the TREC Clueweb collection, we show that this approach improves the retrieval performances. This development could lead to new strategies to address the fusion of relevance estimates provided by different systems.

AB - Recently, mean-variance analysis has been proposed as a novel paradigm to model document ranking in Information Retrieval. The main merit of this approach is that it diversifies the ranking of retrieved documents. In its original formulation, the strategy considers both the mean of relevance estimates of retrieved documents and their variance. However, when this strategy has been empirically instantiated, the concepts of mean and variance are discarded in favour of a point-wise estimation of relevance (to replace the mean) and of a parameter to be tuned or, alternatively, a quantity dependent upon the document length (to replace the variance). In this paper we revisit this ranking strategy by going back to its roots: mean and variance. For each retrieved document, we infer a relevance distribution from a series of point-wise relevance estimations provided by a number of different systems. This is used to compute the mean and the variance of document relevance estimates. On the TREC Clueweb collection, we show that this approach improves the retrieval performances. This development could lead to new strategies to address the fusion of relevance estimates provided by different systems.

KW - information retrieval

KW - mean variance analysis

UR - http://dl.acm.org/citation.cfm?id=1996889.1996986

M3 - Conference contribution book

SN - 978-3-642-20160-8

T3 - Lecture Notes in Computer Science

SP - 716

EP - 720

BT - ECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval

PB - Springer-Verlag

CY - Berlin, Heidelberg

ER -

Zuccon G, Azzopardi L, van Rijsbergen K. Back to the roots: mean-variance analysis of relevance estimations. In ECIR'11 Proceedings of the 33rd European Conference on Advances in Information Retrieval. Berlin, Heidelberg: Springer-Verlag. 2011. p. 716-720. (Lecture Notes in Computer Science).