Query quality: user ratings and system predictions

Claudia Hauff; Franciska de Jong; Diane Kelly; Leif Azzopardi

doi:10.1145/1835449.1835594

Query quality: user ratings and system predictions

Claudia Hauff, Franciska de Jong, Diane Kelly, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

5 Citations (Scopus)

Abstract

Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.

Original language	English
Title of host publication	SIGIR '10 Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval
Place of Publication	New York, NY, USA
Pages	743-744
Number of pages	2
DOIs	https://doi.org/10.1145/1835449.1835594
Publication status	Published - 19 Jul 2010
Externally published	Yes

Keywords

query performance prediction
information retrieval

Access to Document

10.1145/1835449.1835594

Cite this

@inproceedings{ce9762e9778540538ecd75041b8f851e,

title = "Query quality: user ratings and system predictions",

abstract = "Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.",

keywords = "query performance prediction, information retrieval",

author = "Claudia Hauff and {de Jong}, Franciska and Diane Kelly and Leif Azzopardi",

year = "2010",

month = jul,

day = "19",

doi = "10.1145/1835449.1835594",

language = "English",

isbn = "978-1-4503-0153-4",

pages = "743--744",

booktitle = "SIGIR '10 Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval",

}

TY - GEN

T1 - Query quality

T2 - user ratings and system predictions

AU - Hauff, Claudia

AU - de Jong, Franciska

AU - Kelly, Diane

AU - Azzopardi, Leif

PY - 2010/7/19

Y1 - 2010/7/19

N2 - Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.

AB - Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.

KW - query performance prediction

KW - information retrieval

U2 - 10.1145/1835449.1835594

DO - 10.1145/1835449.1835594

M3 - Conference contribution book

SN - 978-1-4503-0153-4

SP - 743

EP - 744

BT - SIGIR '10 Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval

CY - New York, NY, USA

ER -

Query quality: user ratings and system predictions

Abstract

Keywords

Access to Document

Fingerprint

Cite this