Retrievability: an evaluation measure for higher order information access tasks

Leif Azzopardi, Vishwa Vinay

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

70 Citations (Scopus)


Evaluation in Information Retrieval (IR) has long focused on effectiveness and efficiency. However, new and emerging access tasks now demand alternative evaluation measures which go beyond this traditional view. A retrieval system provides a means of gaining access to documents, therefore intuitively, our view of the collection is shaped by the retrieval system. In this paper, we outline some emerging information access related scenarios that require knowledge about how the retrieval system affects the users' ability to access information. This provides the motivation for the proposed evaluation measures and methodology where the focus is on capturing the behavior of the system, in terms of how retrievable it makes individual documents within the collection. To demonstrate the utility of the proposed methods, we perform an extensive analysis on two TREC collections showing how the measures can be applied to evaluate different information access questions. For higher order information access tasks that are inherently dependent on retrievability, our novel evaluation methodology emphasizes that effectiveness is an insufficient characterization of a retrieval system. This paper provides the foundations for the evaluation of higher order access related tasks.
Original languageEnglish
Title of host publicationCIKM '08 Proceedings of the 17th ACM Conference on Information and Knowledge Management
Place of PublicationNew York, NY, USA
Number of pages10
Publication statusPublished - 26 Oct 2008
Externally publishedYes


  • accessibililty
  • retrievability


Dive into the research topics of 'Retrievability: an evaluation measure for higher order information access tasks'. Together they form a unique fingerprint.

Cite this