Methods for ranking information retrieval systems without relevance judgements

S. Wu, F. Crestani

Research output: Contribution to conferencePaper

Abstract

In this paper we present some new methods of ranking information retrieval systems without relevance judgement. The common ground of these methods is using a measure we called reference count . An extensive experimentation was conducted to evaluate the effectiveness of the proposed methods using various different standards Information Retrieval evaluation measures for the ranking, like average precision, R-precision, and precision and different document levels. We also compared the effectiveness of the proposed methods with the method proposed by Soboroff et al. The experimental results showed that the proposed methods are effective, and in many cases are more effective than Soboroff at al.'s method.
LanguageEnglish
Pages811-816
Number of pages5
Publication statusPublished - Mar 2003
EventProceedings of the 2003 ACM Symposium on Applied Computing - Melbourne, Florida, USA
Duration: 9 Mar 200312 Mar 2003

Conference

ConferenceProceedings of the 2003 ACM Symposium on Applied Computing
CityMelbourne, Florida, USA
Period9/03/0312/03/03

Fingerprint

Information retrieval systems
Information retrieval

Keywords

  • information retrieval
  • ranking
  • relevance judgement
  • performance evaluation

Cite this

Wu, S., & Crestani, F. (2003). Methods for ranking information retrieval systems without relevance judgements. 811-816. Paper presented at Proceedings of the 2003 ACM Symposium on Applied Computing, Melbourne, Florida, USA, .
Wu, S. ; Crestani, F. / Methods for ranking information retrieval systems without relevance judgements. Paper presented at Proceedings of the 2003 ACM Symposium on Applied Computing, Melbourne, Florida, USA, .5 p.
@conference{3bd8e1116f56419a893e7c8c09f86553,
title = "Methods for ranking information retrieval systems without relevance judgements",
abstract = "In this paper we present some new methods of ranking information retrieval systems without relevance judgement. The common ground of these methods is using a measure we called reference count . An extensive experimentation was conducted to evaluate the effectiveness of the proposed methods using various different standards Information Retrieval evaluation measures for the ranking, like average precision, R-precision, and precision and different document levels. We also compared the effectiveness of the proposed methods with the method proposed by Soboroff et al. The experimental results showed that the proposed methods are effective, and in many cases are more effective than Soboroff at al.'s method.",
keywords = "information retrieval, ranking, relevance judgement, performance evaluation",
author = "S. Wu and F. Crestani",
year = "2003",
month = "3",
language = "English",
pages = "811--816",
note = "Proceedings of the 2003 ACM Symposium on Applied Computing ; Conference date: 09-03-2003 Through 12-03-2003",

}

Wu, S & Crestani, F 2003, 'Methods for ranking information retrieval systems without relevance judgements' Paper presented at Proceedings of the 2003 ACM Symposium on Applied Computing, Melbourne, Florida, USA, 9/03/03 - 12/03/03, pp. 811-816.

Methods for ranking information retrieval systems without relevance judgements. / Wu, S.; Crestani, F.

2003. 811-816 Paper presented at Proceedings of the 2003 ACM Symposium on Applied Computing, Melbourne, Florida, USA, .

Research output: Contribution to conferencePaper

TY - CONF

T1 - Methods for ranking information retrieval systems without relevance judgements

AU - Wu, S.

AU - Crestani, F.

PY - 2003/3

Y1 - 2003/3

N2 - In this paper we present some new methods of ranking information retrieval systems without relevance judgement. The common ground of these methods is using a measure we called reference count . An extensive experimentation was conducted to evaluate the effectiveness of the proposed methods using various different standards Information Retrieval evaluation measures for the ranking, like average precision, R-precision, and precision and different document levels. We also compared the effectiveness of the proposed methods with the method proposed by Soboroff et al. The experimental results showed that the proposed methods are effective, and in many cases are more effective than Soboroff at al.'s method.

AB - In this paper we present some new methods of ranking information retrieval systems without relevance judgement. The common ground of these methods is using a measure we called reference count . An extensive experimentation was conducted to evaluate the effectiveness of the proposed methods using various different standards Information Retrieval evaluation measures for the ranking, like average precision, R-precision, and precision and different document levels. We also compared the effectiveness of the proposed methods with the method proposed by Soboroff et al. The experimental results showed that the proposed methods are effective, and in many cases are more effective than Soboroff at al.'s method.

KW - information retrieval

KW - ranking

KW - relevance judgement

KW - performance evaluation

UR - http://www.cis.strath.ac.uk/research/publications/papers/strath_cis_publication_31.pdf

M3 - Paper

SP - 811

EP - 816

ER -

Wu S, Crestani F. Methods for ranking information retrieval systems without relevance judgements. 2003. Paper presented at Proceedings of the 2003 ACM Symposium on Applied Computing, Melbourne, Florida, USA, .