The relative effects of knowledge, interest and confidence in assessing relevance

I. Ruthven, M. Baillie, D. Elsweiler

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and their confidence in assessing relevance for a topic, affect the relevance judgements made and the assessor's ability to predict which documents they will assess as being relevant. This study found that each of the three factors (interest, knowledge and confidence) had an affect on how many documents were assessed as relevant and the balance between how many documents were marked as marginally or highly relevant. Also these factors are shown to affect an assessors' ability to predict what information they will finally mark as being relevant.
Original languageEnglish
Pages (from-to)482-504
Number of pages22
JournalJournal of Documentation
Volume63
Issue number4
DOIs
Publication statusPublished - 2007

Fingerprint

confidence
ability

Keywords

  • cognition
  • information retrieval
  • information searches
  • search output
  • relevance ranking

Cite this

@article{525811154e1d48da9885d83a751fa523,
title = "The relative effects of knowledge, interest and confidence in assessing relevance",
abstract = "The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and their confidence in assessing relevance for a topic, affect the relevance judgements made and the assessor's ability to predict which documents they will assess as being relevant. This study found that each of the three factors (interest, knowledge and confidence) had an affect on how many documents were assessed as relevant and the balance between how many documents were marked as marginally or highly relevant. Also these factors are shown to affect an assessors' ability to predict what information they will finally mark as being relevant.",
keywords = "cognition, information retrieval, information searches, search output, relevance ranking",
author = "I. Ruthven and M. Baillie and D. Elsweiler",
year = "2007",
doi = "10.1108/00220410710758986",
language = "English",
volume = "63",
pages = "482--504",
journal = "Journal of Documentation",
issn = "0022-0418",
publisher = "Emerald Publishing Limited",
number = "4",

}

The relative effects of knowledge, interest and confidence in assessing relevance. / Ruthven, I.; Baillie, M.; Elsweiler, D.

In: Journal of Documentation, Vol. 63, No. 4, 2007, p. 482-504.

Research output: Contribution to journalArticle

TY - JOUR

T1 - The relative effects of knowledge, interest and confidence in assessing relevance

AU - Ruthven, I.

AU - Baillie, M.

AU - Elsweiler, D.

PY - 2007

Y1 - 2007

N2 - The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and their confidence in assessing relevance for a topic, affect the relevance judgements made and the assessor's ability to predict which documents they will assess as being relevant. This study found that each of the three factors (interest, knowledge and confidence) had an affect on how many documents were assessed as relevant and the balance between how many documents were marked as marginally or highly relevant. Also these factors are shown to affect an assessors' ability to predict what information they will finally mark as being relevant.

AB - The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and their confidence in assessing relevance for a topic, affect the relevance judgements made and the assessor's ability to predict which documents they will assess as being relevant. This study found that each of the three factors (interest, knowledge and confidence) had an affect on how many documents were assessed as relevant and the balance between how many documents were marked as marginally or highly relevant. Also these factors are shown to affect an assessors' ability to predict what information they will finally mark as being relevant.

KW - cognition

KW - information retrieval

KW - information searches

KW - search output

KW - relevance ranking

UR - http://www.cis.strath.ac.uk/research/publications/papers/strath_cis_publication_1409.pdf

UR - http://dx.doi.org/10.1108/00220410710758986

U2 - 10.1108/00220410710758986

DO - 10.1108/00220410710758986

M3 - Article

VL - 63

SP - 482

EP - 504

JO - Journal of Documentation

JF - Journal of Documentation

SN - 0022-0418

IS - 4

ER -