BERT-based transformers for early detection of mental health illnesses

Rodrigo Martínez-Castaño, Amal Htait, Leif Azzopardi, Yashar Moshfeghi

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

8 Citations (Scopus)
102 Downloads (Pure)


This paper briefly describes our research groups' efforts in tackling Task 1 (Early Detection of Signs of Self-Harm), and Task 2 (Measuring the Severity of the Signs of Depression) from the CLEF eRisk Track. Core to how we approached these problems was the use of BERT-based classifiers which were trained specifically for each task. Our results on both tasks indicate that this approach delivers high performance across a series of measures, particularly for Task 1, where our submissions obtained the best performance for precision, F1, latency-weighted F1 and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3% for Task 1. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., anorexia, depression, suicide).

Original languageEnglish
Title of host publicationExperimental IR Meets Multilinguality, Multimodality, and Interaction
Subtitle of host publication12th International Conference of the CLEF Association, CLEF 2021, Proceedings
EditorsK. Selçuk Candan, Bogdan Ionescu, Lorraine Goeuriot, Birger Larsen, Henning Müller, Alexis Joly, Maria Maistro, Florina Piroi, Guglielmo Faggioli, Nicola Ferro
Place of PublicationCham, Switzerland
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages12
ISBN (Print)9783030852504
Publication statusPublished - 14 Sept 2021
Event12th International Conference of the Cross-Language Evaluation Forum for European Languages, CLEF 2021 - Virtual, Online
Duration: 21 Sept 202124 Sept 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12880 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference12th International Conference of the Cross-Language Evaluation Forum for European Languages, CLEF 2021
CityVirtual, Online


  • BERT
  • classification
  • depression
  • early detection
  • self-harm
  • social media


Dive into the research topics of 'BERT-based transformers for early detection of mental health illnesses'. Together they form a unique fingerprint.

Cite this