CrimeBB: enabling cybercrime research on underground forums at scale

Sergio Pastrana, Daniel R. Thomas, Alice Hutchings, Richard Clayton

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

Abstract

Underground forums allow criminals to interact, exchange knowledge, and trade in products and services. They also provide a pathway into cybercrime, tempting the curious to join those already motivated to obtain easy money. Analysing these forums enables us to better understand the behaviours of offenders and pathways into crime. Prior research has been valuable, but limited by a reliance on datasets that are incomplete or outdated. More complete data, going back many years, allows for comprehensive research into the evolution of forums and their users. We describe CrimeBot, a crawler designed around the particular challenges of capturing data from underground forums. CrimeBot is used to update and maintain CrimeBB, a dataset of more than 48m posts made from 1m accounts in 4 different operational forums over a decade. This dataset presents a new opportunity for large-scale and longitudinal analysis using up-to-date information. We illustrate the potential by presenting a case study using CrimeBB, which analyses which activities lead new actors into engagement with cybercrime. CrimeBB is available to other academic researchers under a legal agreement, designed to prevent misuse and provide safeguards for ethical research.
LanguageEnglish
Title of host publicationProceedings of the 2018 World Wide Web Conference - WWW '18
Pages1845-1854
Number of pages10
DOIs
Publication statusPublished - 27 Apr 2018
Event2018 World Wide Web Conference on World Wide Web - WWW 2018 - Lyon, France
Duration: 23 Apr 201827 Apr 2018

Conference

Conference2018 World Wide Web Conference on World Wide Web - WWW 2018
Abbreviated titleWWW 2018
CountryFrance
CityLyon
Period23/04/1827/04/18

Fingerprint

Crime

Keywords

  • information systems
  • underground forums
  • cybercrime
  • money laundering
  • ethics
  • data sharing
  • web crawling
  • CrimeBot
  • CrimeBB

Cite this

Pastrana, S., Thomas, D. R., Hutchings, A., & Clayton, R. (2018). CrimeBB: enabling cybercrime research on underground forums at scale. In Proceedings of the 2018 World Wide Web Conference - WWW '18 (pp. 1845-1854) https://doi.org/10.1145/3178876.3186178, https://doi.org/10.17863/CAM.23339
Pastrana, Sergio ; Thomas, Daniel R. ; Hutchings, Alice ; Clayton, Richard. / CrimeBB : enabling cybercrime research on underground forums at scale. Proceedings of the 2018 World Wide Web Conference - WWW '18. 2018. pp. 1845-1854
@inproceedings{889db532e0204ec0aa20ef4cde47a242,
title = "CrimeBB: enabling cybercrime research on underground forums at scale",
abstract = "Underground forums allow criminals to interact, exchange knowledge, and trade in products and services. They also provide a pathway into cybercrime, tempting the curious to join those already motivated to obtain easy money. Analysing these forums enables us to better understand the behaviours of offenders and pathways into crime. Prior research has been valuable, but limited by a reliance on datasets that are incomplete or outdated. More complete data, going back many years, allows for comprehensive research into the evolution of forums and their users. We describe CrimeBot, a crawler designed around the particular challenges of capturing data from underground forums. CrimeBot is used to update and maintain CrimeBB, a dataset of more than 48m posts made from 1m accounts in 4 different operational forums over a decade. This dataset presents a new opportunity for large-scale and longitudinal analysis using up-to-date information. We illustrate the potential by presenting a case study using CrimeBB, which analyses which activities lead new actors into engagement with cybercrime. CrimeBB is available to other academic researchers under a legal agreement, designed to prevent misuse and provide safeguards for ethical research.",
keywords = "information systems, underground forums, cybercrime, money laundering, ethics, data sharing, web crawling, CrimeBot, CrimeBB",
author = "Sergio Pastrana and Thomas, {Daniel R.} and Alice Hutchings and Richard Clayton",
year = "2018",
month = "4",
day = "27",
doi = "10.1145/3178876.3186178",
language = "English",
isbn = "9781450356398",
pages = "1845--1854",
booktitle = "Proceedings of the 2018 World Wide Web Conference - WWW '18",

}

Pastrana, S, Thomas, DR, Hutchings, A & Clayton, R 2018, CrimeBB: enabling cybercrime research on underground forums at scale. in Proceedings of the 2018 World Wide Web Conference - WWW '18. pp. 1845-1854, 2018 World Wide Web Conference on World Wide Web - WWW 2018, Lyon, France, 23/04/18. https://doi.org/10.1145/3178876.3186178, https://doi.org/10.17863/CAM.23339

CrimeBB : enabling cybercrime research on underground forums at scale. / Pastrana, Sergio; Thomas, Daniel R.; Hutchings, Alice; Clayton, Richard.

Proceedings of the 2018 World Wide Web Conference - WWW '18. 2018. p. 1845-1854.

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - CrimeBB

T2 - enabling cybercrime research on underground forums at scale

AU - Pastrana, Sergio

AU - Thomas, Daniel R.

AU - Hutchings, Alice

AU - Clayton, Richard

PY - 2018/4/27

Y1 - 2018/4/27

N2 - Underground forums allow criminals to interact, exchange knowledge, and trade in products and services. They also provide a pathway into cybercrime, tempting the curious to join those already motivated to obtain easy money. Analysing these forums enables us to better understand the behaviours of offenders and pathways into crime. Prior research has been valuable, but limited by a reliance on datasets that are incomplete or outdated. More complete data, going back many years, allows for comprehensive research into the evolution of forums and their users. We describe CrimeBot, a crawler designed around the particular challenges of capturing data from underground forums. CrimeBot is used to update and maintain CrimeBB, a dataset of more than 48m posts made from 1m accounts in 4 different operational forums over a decade. This dataset presents a new opportunity for large-scale and longitudinal analysis using up-to-date information. We illustrate the potential by presenting a case study using CrimeBB, which analyses which activities lead new actors into engagement with cybercrime. CrimeBB is available to other academic researchers under a legal agreement, designed to prevent misuse and provide safeguards for ethical research.

AB - Underground forums allow criminals to interact, exchange knowledge, and trade in products and services. They also provide a pathway into cybercrime, tempting the curious to join those already motivated to obtain easy money. Analysing these forums enables us to better understand the behaviours of offenders and pathways into crime. Prior research has been valuable, but limited by a reliance on datasets that are incomplete or outdated. More complete data, going back many years, allows for comprehensive research into the evolution of forums and their users. We describe CrimeBot, a crawler designed around the particular challenges of capturing data from underground forums. CrimeBot is used to update and maintain CrimeBB, a dataset of more than 48m posts made from 1m accounts in 4 different operational forums over a decade. This dataset presents a new opportunity for large-scale and longitudinal analysis using up-to-date information. We illustrate the potential by presenting a case study using CrimeBB, which analyses which activities lead new actors into engagement with cybercrime. CrimeBB is available to other academic researchers under a legal agreement, designed to prevent misuse and provide safeguards for ethical research.

KW - information systems

KW - underground forums

KW - cybercrime

KW - money laundering

KW - ethics

KW - data sharing

KW - web crawling

KW - CrimeBot

KW - CrimeBB

U2 - 10.1145/3178876.3186178

DO - 10.1145/3178876.3186178

M3 - Conference contribution book

SN - 9781450356398

SP - 1845

EP - 1854

BT - Proceedings of the 2018 World Wide Web Conference - WWW '18

ER -

Pastrana S, Thomas DR, Hutchings A, Clayton R. CrimeBB: enabling cybercrime research on underground forums at scale. In Proceedings of the 2018 World Wide Web Conference - WWW '18. 2018. p. 1845-1854 https://doi.org/10.1145/3178876.3186178, https://doi.org/10.17863/CAM.23339