Overview of the CLEF eHealth Evaluation Lab 2019

Liadh Kelly; Hanna Suominen; Lorraine Goeuriot; Mariana Neves; Evangelos Kanoulas; Dan Li; Leif Azzopardi; Rene Spijker; Guido Zuccon; Harrisen Scells; João Palotti

doi:10.1007/978-3-030-28577-7_26

Overview of the CLEF eHealth Evaluation Lab 2019

Liadh Kelly^*, Hanna Suominen, Lorraine Goeuriot, Mariana Neves, Evangelos Kanoulas, Dan Li, Leif Azzopardi, Rene Spijker, Guido Zuccon, Harrisen Scells, João Palotti

^*Corresponding author for this work

Computer And Information Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

38 Citations (Scopus)

Abstract

In this paper, we provide an overview of the seventh annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2019 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring electronic health information in a multilingual setting. This year’s lab advertised three tasks: Task 1 on indexing non-technical summaries of German animal experiments with International Classification of Diseases, Version 10 codes; Task 2 on technology assisted reviews in empirical medicine building on 2017 and 2018 tasks in English; and Task 3 on consumer health search in mono- and multilingual settings that builds on the 2013–18 Information Retrieval tasks. In total nine teams took part in these tasks (six in Task 1 and three in Task 2). Herein, we describe the resources created for these tasks and evaluation methodology adopted. We also provide a brief summary of participants of this year’s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

Original language	English
Title of host publication	Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings
Editors	Fabio Crestani, Martin Braschler, Jacques Savoy, Andreas Rauber, Henning Müller, David E. Losada, Gundula Heinatz Bürki, Linda Cappellato, Nicola Ferro
Place of Publication	Cham
Publisher	Springer-Verlag
Pages	322-339
Number of pages	18
ISBN (Electronic)	9783030285777
ISBN (Print)	9783030285760
DOIs	https://doi.org/10.1007/978-3-030-28577-7_26
Publication status	E-pub ahead of print - 3 Aug 2019
Event	10th International Conference of the CLEF Association, CLEF 2019 - Lugano, Switzerland Duration: 9 Sept 2019 → 12 Sept 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11696 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	10th International Conference of the CLEF Association, CLEF 2019
Country/Territory	Switzerland
City	Lugano
Period	9/09/19 → 12/09/19

Keywords

entity linking
evaluation
health records
high recall
information extraction
information retrieval
medical informatics
self-diagnosis
systematic reviews
test-set generation
text classification
text segmentation

Access to Document

10.1007/978-3-030-28577-7_26

Cite this

Kelly, L., Suominen, H., Goeuriot, L., Neves, M., Kanoulas, E., Li, D., Azzopardi, L., Spijker, R., Zuccon, G., Scells, H., & Palotti, J. (2019). Overview of the CLEF eHealth Evaluation Lab 2019. In F. Crestani, M. Braschler, J. Savoy, A. Rauber, H. Müller, D. E. Losada, G. Heinatz Bürki, L. Cappellato, & N. Ferro (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings (pp. 322-339). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11696 LNCS). Springer-Verlag. Advance online publication. https://doi.org/10.1007/978-3-030-28577-7_26

Kelly, Liadh ; Suominen, Hanna ; Goeuriot, Lorraine et al. / Overview of the CLEF eHealth Evaluation Lab 2019. Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings. editor / Fabio Crestani ; Martin Braschler ; Jacques Savoy ; Andreas Rauber ; Henning Müller ; David E. Losada ; Gundula Heinatz Bürki ; Linda Cappellato ; Nicola Ferro. Cham : Springer-Verlag, 2019. pp. 322-339 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{dab1f0e70d72433fa8a717e729a425f9,

title = "Overview of the CLEF eHealth Evaluation Lab 2019",

abstract = "In this paper, we provide an overview of the seventh annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2019 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring electronic health information in a multilingual setting. This year{\textquoteright}s lab advertised three tasks: Task 1 on indexing non-technical summaries of German animal experiments with International Classification of Diseases, Version 10 codes; Task 2 on technology assisted reviews in empirical medicine building on 2017 and 2018 tasks in English; and Task 3 on consumer health search in mono- and multilingual settings that builds on the 2013–18 Information Retrieval tasks. In total nine teams took part in these tasks (six in Task 1 and three in Task 2). Herein, we describe the resources created for these tasks and evaluation methodology adopted. We also provide a brief summary of participants of this year{\textquoteright}s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.",

keywords = "entity linking, evaluation, health records, high recall, information extraction, information retrieval, medical informatics, self-diagnosis, systematic reviews, test-set generation, text classification, text segmentation",

author = "Liadh Kelly and Hanna Suominen and Lorraine Goeuriot and Mariana Neves and Evangelos Kanoulas and Dan Li and Leif Azzopardi and Rene Spijker and Guido Zuccon and Harrisen Scells and Jo{\~a}o Palotti",

note = "Kelly, L. et al. (2019). Overview of the CLEF eHealth Evaluation Lab 2019. In: , et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2019. Lecture Notes in Computer Science(), vol 11696. Springer, Cham. https://doi.org/10.1007/978-3-030-28577-7_26; 10th International Conference of the CLEF Association, CLEF 2019 ; Conference date: 09-09-2019 Through 12-09-2019",

year = "2019",

month = aug,

day = "3",

doi = "10.1007/978-3-030-28577-7_26",

language = "English",

isbn = "9783030285760",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag",

pages = "322--339",

editor = "Fabio Crestani and Martin Braschler and Jacques Savoy and Andreas Rauber and Henning M{\"u}ller and Losada, {David E.} and {Heinatz B{\"u}rki}, Gundula and Linda Cappellato and Nicola Ferro",

booktitle = "Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings",

}

Kelly, L, Suominen, H, Goeuriot, L, Neves, M, Kanoulas, E, Li, D, Azzopardi, L, Spijker, R, Zuccon, G, Scells, H & Palotti, J 2019, Overview of the CLEF eHealth Evaluation Lab 2019. in F Crestani, M Braschler, J Savoy, A Rauber, H Müller, DE Losada, G Heinatz Bürki, L Cappellato & N Ferro (eds), Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11696 LNCS, Springer-Verlag, Cham, pp. 322-339, 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, 9/09/19. https://doi.org/10.1007/978-3-030-28577-7_26

Overview of the CLEF eHealth Evaluation Lab 2019. / Kelly, Liadh; Suominen, Hanna; Goeuriot, Lorraine et al.
Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings. ed. / Fabio Crestani; Martin Braschler; Jacques Savoy; Andreas Rauber; Henning Müller; David E. Losada; Gundula Heinatz Bürki; Linda Cappellato; Nicola Ferro. Cham: Springer-Verlag, 2019. p. 322-339 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11696 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

TY - GEN

T1 - Overview of the CLEF eHealth Evaluation Lab 2019

AU - Kelly, Liadh

AU - Suominen, Hanna

AU - Goeuriot, Lorraine

AU - Neves, Mariana

AU - Kanoulas, Evangelos

AU - Li, Dan

AU - Azzopardi, Leif

AU - Spijker, Rene

AU - Zuccon, Guido

AU - Scells, Harrisen

AU - Palotti, João

N1 - Kelly, L. et al. (2019). Overview of the CLEF eHealth Evaluation Lab 2019. In: , et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2019. Lecture Notes in Computer Science(), vol 11696. Springer, Cham. https://doi.org/10.1007/978-3-030-28577-7_26

PY - 2019/8/3

Y1 - 2019/8/3

N2 - In this paper, we provide an overview of the seventh annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2019 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring electronic health information in a multilingual setting. This year’s lab advertised three tasks: Task 1 on indexing non-technical summaries of German animal experiments with International Classification of Diseases, Version 10 codes; Task 2 on technology assisted reviews in empirical medicine building on 2017 and 2018 tasks in English; and Task 3 on consumer health search in mono- and multilingual settings that builds on the 2013–18 Information Retrieval tasks. In total nine teams took part in these tasks (six in Task 1 and three in Task 2). Herein, we describe the resources created for these tasks and evaluation methodology adopted. We also provide a brief summary of participants of this year’s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

AB - In this paper, we provide an overview of the seventh annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2019 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring electronic health information in a multilingual setting. This year’s lab advertised three tasks: Task 1 on indexing non-technical summaries of German animal experiments with International Classification of Diseases, Version 10 codes; Task 2 on technology assisted reviews in empirical medicine building on 2017 and 2018 tasks in English; and Task 3 on consumer health search in mono- and multilingual settings that builds on the 2013–18 Information Retrieval tasks. In total nine teams took part in these tasks (six in Task 1 and three in Task 2). Herein, we describe the resources created for these tasks and evaluation methodology adopted. We also provide a brief summary of participants of this year’s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

KW - entity linking

KW - evaluation

KW - health records

KW - high recall

KW - information extraction

KW - information retrieval

KW - medical informatics

KW - self-diagnosis

KW - systematic reviews

KW - test-set generation

KW - text classification

KW - text segmentation

UR - http://www.scopus.com/inward/record.url?scp=85070529244&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-28577-7_26

DO - 10.1007/978-3-030-28577-7_26

M3 - Conference contribution book

AN - SCOPUS:85070529244

SN - 9783030285760

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 322

EP - 339

BT - Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings

A2 - Crestani, Fabio

A2 - Braschler, Martin

A2 - Savoy, Jacques

A2 - Rauber, Andreas

A2 - Müller, Henning

A2 - Losada, David E.

A2 - Heinatz Bürki, Gundula

A2 - Cappellato, Linda

A2 - Ferro, Nicola

PB - Springer-Verlag

CY - Cham

T2 - 10th International Conference of the CLEF Association, CLEF 2019

Y2 - 9 September 2019 through 12 September 2019

ER -

Kelly L, Suominen H, Goeuriot L, Neves M, Kanoulas E, Li D et al. Overview of the CLEF eHealth Evaluation Lab 2019. In Crestani F, Braschler M, Savoy J, Rauber A, Müller H, Losada DE, Heinatz Bürki G, Cappellato L, Ferro N, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Proceedings. Cham: Springer-Verlag. 2019. p. 322-339. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2019 Aug 3. doi: 10.1007/978-3-030-28577-7_26

Overview of the CLEF eHealth Evaluation Lab 2019

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this