SCAMP

a tool for conducting interactive information retrieval experiments

Gareth Renaud, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

9 Citations (Scopus)

Abstract

Conducting Interactive Information Retrieval (IIR) research is often seen as an arduous and tedious process with a high barrier to entry. This high barrier is due to the overheads in developing and setting up even a simple lab-based IIR experiment. SCAMP (Search ConfigurAtor for experiMenting with PuppyIR) is a web-based tool that we have developed which enables researchers to configure standard IIR experiments. SCAMP provides the infrastructure that handles the major processes within the experimental flow (such as Participant Registration, Consent, Surveys, and the logging and tracking of tasks and participants through the experiment). Consequently, SCAMP reduces the time required to create an experiment and is ideal for undergraduate and masters students who would like to conduct an IIR experiment without extensive development. Furthermore, the tool is extensible. Other features can be easily added to SCAMP to customise the experiments. To evaluate SCAMP, we performed a within-subjects experiment where 48 participants used a web search engine with different search aids (query suggestion, spell correction, etc) to complete various web search tasks - to increase the difficulty of the search tasks certain query terms were banned. We use this evaluation to showcase the different features of the SCAMP system and report on how participants perform under difficult querying conditions.
Original languageEnglish
Title of host publicationIIIX '12 Proceedings of the 4th Information Interaction in Context Symposium
Place of PublicationNew York, NY, USA
Pages286-289
Number of pages4
DOIs
Publication statusPublished - 21 Aug 2012
Externally publishedYes

Fingerprint

information retrieval
experiment
search engine
infrastructure
evaluation
student

Keywords

  • analyse
  • taboo search
  • online experiments
  • information retrieval

Cite this

Renaud, G., & Azzopardi, L. (2012). SCAMP: a tool for conducting interactive information retrieval experiments. In IIIX '12 Proceedings of the 4th Information Interaction in Context Symposium (pp. 286-289). New York, NY, USA. https://doi.org/10.1145/2362724.2362776
Renaud, Gareth ; Azzopardi, Leif. / SCAMP : a tool for conducting interactive information retrieval experiments. IIIX '12 Proceedings of the 4th Information Interaction in Context Symposium. New York, NY, USA, 2012. pp. 286-289
@inproceedings{16dd90f12d46465e8eb64e82623d0333,
title = "SCAMP: a tool for conducting interactive information retrieval experiments",
abstract = "Conducting Interactive Information Retrieval (IIR) research is often seen as an arduous and tedious process with a high barrier to entry. This high barrier is due to the overheads in developing and setting up even a simple lab-based IIR experiment. SCAMP (Search ConfigurAtor for experiMenting with PuppyIR) is a web-based tool that we have developed which enables researchers to configure standard IIR experiments. SCAMP provides the infrastructure that handles the major processes within the experimental flow (such as Participant Registration, Consent, Surveys, and the logging and tracking of tasks and participants through the experiment). Consequently, SCAMP reduces the time required to create an experiment and is ideal for undergraduate and masters students who would like to conduct an IIR experiment without extensive development. Furthermore, the tool is extensible. Other features can be easily added to SCAMP to customise the experiments. To evaluate SCAMP, we performed a within-subjects experiment where 48 participants used a web search engine with different search aids (query suggestion, spell correction, etc) to complete various web search tasks - to increase the difficulty of the search tasks certain query terms were banned. We use this evaluation to showcase the different features of the SCAMP system and report on how participants perform under difficult querying conditions.",
keywords = "analyse, taboo search, online experiments, information retrieval",
author = "Gareth Renaud and Leif Azzopardi",
year = "2012",
month = "8",
day = "21",
doi = "10.1145/2362724.2362776",
language = "English",
isbn = "978-1-4503-1282-0",
pages = "286--289",
booktitle = "IIIX '12 Proceedings of the 4th Information Interaction in Context Symposium",

}

Renaud, G & Azzopardi, L 2012, SCAMP: a tool for conducting interactive information retrieval experiments. in IIIX '12 Proceedings of the 4th Information Interaction in Context Symposium. New York, NY, USA, pp. 286-289. https://doi.org/10.1145/2362724.2362776

SCAMP : a tool for conducting interactive information retrieval experiments. / Renaud, Gareth; Azzopardi, Leif.

IIIX '12 Proceedings of the 4th Information Interaction in Context Symposium. New York, NY, USA, 2012. p. 286-289.

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - SCAMP

T2 - a tool for conducting interactive information retrieval experiments

AU - Renaud, Gareth

AU - Azzopardi, Leif

PY - 2012/8/21

Y1 - 2012/8/21

N2 - Conducting Interactive Information Retrieval (IIR) research is often seen as an arduous and tedious process with a high barrier to entry. This high barrier is due to the overheads in developing and setting up even a simple lab-based IIR experiment. SCAMP (Search ConfigurAtor for experiMenting with PuppyIR) is a web-based tool that we have developed which enables researchers to configure standard IIR experiments. SCAMP provides the infrastructure that handles the major processes within the experimental flow (such as Participant Registration, Consent, Surveys, and the logging and tracking of tasks and participants through the experiment). Consequently, SCAMP reduces the time required to create an experiment and is ideal for undergraduate and masters students who would like to conduct an IIR experiment without extensive development. Furthermore, the tool is extensible. Other features can be easily added to SCAMP to customise the experiments. To evaluate SCAMP, we performed a within-subjects experiment where 48 participants used a web search engine with different search aids (query suggestion, spell correction, etc) to complete various web search tasks - to increase the difficulty of the search tasks certain query terms were banned. We use this evaluation to showcase the different features of the SCAMP system and report on how participants perform under difficult querying conditions.

AB - Conducting Interactive Information Retrieval (IIR) research is often seen as an arduous and tedious process with a high barrier to entry. This high barrier is due to the overheads in developing and setting up even a simple lab-based IIR experiment. SCAMP (Search ConfigurAtor for experiMenting with PuppyIR) is a web-based tool that we have developed which enables researchers to configure standard IIR experiments. SCAMP provides the infrastructure that handles the major processes within the experimental flow (such as Participant Registration, Consent, Surveys, and the logging and tracking of tasks and participants through the experiment). Consequently, SCAMP reduces the time required to create an experiment and is ideal for undergraduate and masters students who would like to conduct an IIR experiment without extensive development. Furthermore, the tool is extensible. Other features can be easily added to SCAMP to customise the experiments. To evaluate SCAMP, we performed a within-subjects experiment where 48 participants used a web search engine with different search aids (query suggestion, spell correction, etc) to complete various web search tasks - to increase the difficulty of the search tasks certain query terms were banned. We use this evaluation to showcase the different features of the SCAMP system and report on how participants perform under difficult querying conditions.

KW - analyse

KW - taboo search

KW - online experiments

KW - information retrieval

U2 - 10.1145/2362724.2362776

DO - 10.1145/2362724.2362776

M3 - Conference contribution book

SN - 978-1-4503-1282-0

SP - 286

EP - 289

BT - IIIX '12 Proceedings of the 4th Information Interaction in Context Symposium

CY - New York, NY, USA

ER -

Renaud G, Azzopardi L. SCAMP: a tool for conducting interactive information retrieval experiments. In IIIX '12 Proceedings of the 4th Information Interaction in Context Symposium. New York, NY, USA. 2012. p. 286-289 https://doi.org/10.1145/2362724.2362776