Report on the First International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2015)

Laure Soulier, Lynda Tamine, Tetsuya Sakai, Leif Azzopardi, Jeremy Pickens

Research output: Contribution to journalArticle

Abstract

The workshop on the evaluation of collaborative information retrieval and seeking (ECol) was held in conjunction with the 24th Conference on Information and Knowledge Management (CIKM) in Melbourne, Australia. The workshop featured three main elements. First, a keynote on the main dimensions, challenges, and opportunities in collaborative information retrieval and seeking by Chirag Shah. Second, an oral presentation session in which four papers were presented. Third, a discussion based on three seed research questions: (1) In what ways is collaborative search evaluation more challenging than individual interactive information retrieval (IIIR) evaluation? (2) Would it be possible and/or useful to standardise experimental designs and data for collaborative search evaluation? and (3) For evaluating collaborative search, can we leverage ideas from other tasks such as diversified search, subtopic mining and/or e-discovery? The discussion was intense and raised many points and issues, leading to the proposition that a new evaluation track focused on collaborative information retrieval/seeking tasks, would be worthwhile.
LanguageEnglish
Pages42-48
Number of pages7
JournalACM SIGIR Forum
Volume50
Issue number1
DOIs
Publication statusPublished - 1 Jun 2016

Fingerprint

Information retrieval
Knowledge management
Design of experiments
Information management
Seed
Information seeking
Evaluation

Keywords

  • information seeking
  • information retieval

Cite this

Soulier, Laure ; Tamine, Lynda ; Sakai, Tetsuya ; Azzopardi, Leif ; Pickens, Jeremy. / Report on the First International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2015). In: ACM SIGIR Forum. 2016 ; Vol. 50, No. 1. pp. 42-48.
@article{fd3ea3b4be8644bb9a2b1c3fb1c60de0,
title = "Report on the First International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2015)",
abstract = "The workshop on the evaluation of collaborative information retrieval and seeking (ECol) was held in conjunction with the 24th Conference on Information and Knowledge Management (CIKM) in Melbourne, Australia. The workshop featured three main elements. First, a keynote on the main dimensions, challenges, and opportunities in collaborative information retrieval and seeking by Chirag Shah. Second, an oral presentation session in which four papers were presented. Third, a discussion based on three seed research questions: (1) In what ways is collaborative search evaluation more challenging than individual interactive information retrieval (IIIR) evaluation? (2) Would it be possible and/or useful to standardise experimental designs and data for collaborative search evaluation? and (3) For evaluating collaborative search, can we leverage ideas from other tasks such as diversified search, subtopic mining and/or e-discovery? The discussion was intense and raised many points and issues, leading to the proposition that a new evaluation track focused on collaborative information retrieval/seeking tasks, would be worthwhile.",
keywords = "information seeking, information retieval",
author = "Laure Soulier and Lynda Tamine and Tetsuya Sakai and Leif Azzopardi and Jeremy Pickens",
year = "2016",
month = "6",
day = "1",
doi = "10.1145/2964797.2964805",
language = "English",
volume = "50",
pages = "42--48",
journal = "ACM SIGIR Forum",
issn = "1558-0229",
number = "1",

}

Report on the First International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2015). / Soulier, Laure; Tamine, Lynda; Sakai, Tetsuya; Azzopardi, Leif; Pickens, Jeremy.

In: ACM SIGIR Forum, Vol. 50, No. 1, 01.06.2016, p. 42-48.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Report on the First International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2015)

AU - Soulier, Laure

AU - Tamine, Lynda

AU - Sakai, Tetsuya

AU - Azzopardi, Leif

AU - Pickens, Jeremy

PY - 2016/6/1

Y1 - 2016/6/1

N2 - The workshop on the evaluation of collaborative information retrieval and seeking (ECol) was held in conjunction with the 24th Conference on Information and Knowledge Management (CIKM) in Melbourne, Australia. The workshop featured three main elements. First, a keynote on the main dimensions, challenges, and opportunities in collaborative information retrieval and seeking by Chirag Shah. Second, an oral presentation session in which four papers were presented. Third, a discussion based on three seed research questions: (1) In what ways is collaborative search evaluation more challenging than individual interactive information retrieval (IIIR) evaluation? (2) Would it be possible and/or useful to standardise experimental designs and data for collaborative search evaluation? and (3) For evaluating collaborative search, can we leverage ideas from other tasks such as diversified search, subtopic mining and/or e-discovery? The discussion was intense and raised many points and issues, leading to the proposition that a new evaluation track focused on collaborative information retrieval/seeking tasks, would be worthwhile.

AB - The workshop on the evaluation of collaborative information retrieval and seeking (ECol) was held in conjunction with the 24th Conference on Information and Knowledge Management (CIKM) in Melbourne, Australia. The workshop featured three main elements. First, a keynote on the main dimensions, challenges, and opportunities in collaborative information retrieval and seeking by Chirag Shah. Second, an oral presentation session in which four papers were presented. Third, a discussion based on three seed research questions: (1) In what ways is collaborative search evaluation more challenging than individual interactive information retrieval (IIIR) evaluation? (2) Would it be possible and/or useful to standardise experimental designs and data for collaborative search evaluation? and (3) For evaluating collaborative search, can we leverage ideas from other tasks such as diversified search, subtopic mining and/or e-discovery? The discussion was intense and raised many points and issues, leading to the proposition that a new evaluation track focused on collaborative information retrieval/seeking tasks, would be worthwhile.

KW - information seeking

KW - information retieval

UR - https://www.irit.fr/ECol2015/

U2 - 10.1145/2964797.2964805

DO - 10.1145/2964797.2964805

M3 - Article

VL - 50

SP - 42

EP - 48

JO - ACM SIGIR Forum

T2 - ACM SIGIR Forum

JF - ACM SIGIR Forum

SN - 1558-0229

IS - 1

ER -