Replication's role in software engineering

Murray Wood, Marc Roper, J. Miller, J. Daly, Andrew Brooks

Research output: Chapter in Book/Report/Conference proceedingChapter

34 Citations (Scopus)

Abstract

We provide motivation for researchers to replicate experiments in software engineering. The ideology of replication is discussed. We address the question: Is an experiment worth repeating? The current lack of replication studies is highlighted. We make clear that exact replication is unattainable and we draw on our first experience of performing an external replication. To categorise various kinds of replication, we propose a simple extension to Basili et al's framework for experimentation in software engineering. We present guidance as to the level of reported detail required to enable others perform a replication. Our conclusion is that there is only one route for empirical software engineering to follow: to make available laboratory packages of experimental materials to facilitate internal and
external replications, especially the latter, which have greater confirming power.
Original languageEnglish
Title of host publicationGuide to Advanced Empirical Software Engineering
EditorsForrest Schull, Janice Singer, Daj Sjoberg
PublisherSpringer
Pages365-379
Number of pages15
ISBN (Print)9781848000438
DOIs
Publication statusPublished - 2008

Fingerprint

Software engineering
Experiments

Keywords

  • empirical
  • reproducible
  • study replication
  • repeatable
  • experimental

Cite this

Wood, M., Roper, M., Miller, J., Daly, J., & Brooks, A. (2008). Replication's role in software engineering. In F. Schull, J. Singer, & D. Sjoberg (Eds.), Guide to Advanced Empirical Software Engineering (pp. 365-379). Springer. https://doi.org/10.1007/978-1-84800-044-5_14
Wood, Murray ; Roper, Marc ; Miller, J. ; Daly, J. ; Brooks, Andrew. / Replication's role in software engineering. Guide to Advanced Empirical Software Engineering . editor / Forrest Schull ; Janice Singer ; Daj Sjoberg. Springer, 2008. pp. 365-379
@inbook{eb01e8d6594841e3bf4c81e54837b842,
title = "Replication's role in software engineering",
abstract = "We provide motivation for researchers to replicate experiments in software engineering. The ideology of replication is discussed. We address the question: Is an experiment worth repeating? The current lack of replication studies is highlighted. We make clear that exact replication is unattainable and we draw on our first experience of performing an external replication. To categorise various kinds of replication, we propose a simple extension to Basili et al's framework for experimentation in software engineering. We present guidance as to the level of reported detail required to enable others perform a replication. Our conclusion is that there is only one route for empirical software engineering to follow: to make available laboratory packages of experimental materials to facilitate internal andexternal replications, especially the latter, which have greater confirming power.",
keywords = "empirical, reproducible, study replication, repeatable, experimental",
author = "Murray Wood and Marc Roper and J. Miller and J. Daly and Andrew Brooks",
year = "2008",
doi = "10.1007/978-1-84800-044-5_14",
language = "English",
isbn = "9781848000438",
pages = "365--379",
editor = "Forrest Schull and Singer, {Janice } and Daj Sjoberg",
booktitle = "Guide to Advanced Empirical Software Engineering",
publisher = "Springer",

}

Wood, M, Roper, M, Miller, J, Daly, J & Brooks, A 2008, Replication's role in software engineering. in F Schull, J Singer & D Sjoberg (eds), Guide to Advanced Empirical Software Engineering . Springer, pp. 365-379. https://doi.org/10.1007/978-1-84800-044-5_14

Replication's role in software engineering. / Wood, Murray; Roper, Marc; Miller, J.; Daly, J.; Brooks, Andrew.

Guide to Advanced Empirical Software Engineering . ed. / Forrest Schull; Janice Singer; Daj Sjoberg. Springer, 2008. p. 365-379.

Research output: Chapter in Book/Report/Conference proceedingChapter

TY - CHAP

T1 - Replication's role in software engineering

AU - Wood, Murray

AU - Roper, Marc

AU - Miller, J.

AU - Daly, J.

AU - Brooks, Andrew

PY - 2008

Y1 - 2008

N2 - We provide motivation for researchers to replicate experiments in software engineering. The ideology of replication is discussed. We address the question: Is an experiment worth repeating? The current lack of replication studies is highlighted. We make clear that exact replication is unattainable and we draw on our first experience of performing an external replication. To categorise various kinds of replication, we propose a simple extension to Basili et al's framework for experimentation in software engineering. We present guidance as to the level of reported detail required to enable others perform a replication. Our conclusion is that there is only one route for empirical software engineering to follow: to make available laboratory packages of experimental materials to facilitate internal andexternal replications, especially the latter, which have greater confirming power.

AB - We provide motivation for researchers to replicate experiments in software engineering. The ideology of replication is discussed. We address the question: Is an experiment worth repeating? The current lack of replication studies is highlighted. We make clear that exact replication is unattainable and we draw on our first experience of performing an external replication. To categorise various kinds of replication, we propose a simple extension to Basili et al's framework for experimentation in software engineering. We present guidance as to the level of reported detail required to enable others perform a replication. Our conclusion is that there is only one route for empirical software engineering to follow: to make available laboratory packages of experimental materials to facilitate internal andexternal replications, especially the latter, which have greater confirming power.

KW - empirical

KW - reproducible

KW - study replication

KW - repeatable

KW - experimental

U2 - 10.1007/978-1-84800-044-5_14

DO - 10.1007/978-1-84800-044-5_14

M3 - Chapter

SN - 9781848000438

SP - 365

EP - 379

BT - Guide to Advanced Empirical Software Engineering

A2 - Schull, Forrest

A2 - Singer, Janice

A2 - Sjoberg, Daj

PB - Springer

ER -

Wood M, Roper M, Miller J, Daly J, Brooks A. Replication's role in software engineering. In Schull F, Singer J, Sjoberg D, editors, Guide to Advanced Empirical Software Engineering . Springer. 2008. p. 365-379 https://doi.org/10.1007/978-1-84800-044-5_14