The use of relevance criteria during predictive judgment: an eye tracking approach

Panos Balatsoukas, Ian Ruthven

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

7 Citations (Scopus)


This paper reports on the findings of a user study that explored how searchers tend to fixate on information associated with different relevance criteria in the web during the relevance judgment process. The user study involved the completion of questionnaires, use of eye tracking technology, talk aloud protocols and post-search interviews. As opposed to previous studies, the present research asked participants to search for real information needs that represented different search contexts (e.g. from searches about personal interest to academic related searches). This permitted the identification of several relevance criteria that naturally occur across different search contexts and the emergence of some fixation patterns, not observed before, associated to the use of these criteria. Although the study examined participants' eye movements for both predictive and evaluative relevance judgments, this paper is focused on the findings of the predictive relevance judgment process and specifically participants' evaluation of the results presented in the search result interface of the Google search engine. It is anticipated that the findings reported in this paper could shed light on the process of predictive relevance judgment and especially on the problem of relevance criteria use through the use of eye tracking.
Original languageEnglish
Title of host publicationASIS&T '10
Subtitle of host publicationProceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem
Place of PublicationNew York, NY
Number of pages10
Publication statusPublished - 22 Oct 2010


  • eye tracking
  • predictive judgment
  • search engines


Dive into the research topics of 'The use of relevance criteria during predictive judgment: an eye tracking approach'. Together they form a unique fingerprint.

Cite this