An Empirical Evaluation of Defect Detection Techniques

M. Roper, M. Wood, J. Miller

Research output: Contribution to journalArticle

36 Citations (Scopus)

Abstract

This report describes an empirical study comparing three defect detection techniques: (a) code reading by stepwise abstraction, (b) functional testing using equivalence partitioning and boundary value analysis, and (c) structural testing using branch coverage. It is a replication of a study that has been carried out at least four times previously over the last 20 years. This study used 47 student subjects to apply the techniques to small C programs in a fractional factorial experimental design. The major findings of the study are: (a) that the individual techniques are of broadly similar effectiveness in terms of observing failures and finding faults, (b) that the relative effectiveness of the techniques depends on the nature of the program and its faults, (c) these techniques are consistently much more effective when used in combination with each other. These results contribute to a growing body of empirical evidence that supports generally held beliefs about the effectiveness of defect detection techniques in software engineering.
LanguageEnglish
Pages763-775
Number of pages12
JournalInformation and Software Technology
Volume39
Issue number11
DOIs
Publication statusPublished - 1997

Fingerprint

Value engineering
Testing
Design of experiments
Software engineering
Students
Defect detection

Keywords

  • Software testing
  • Code reading
  • Code review
  • Functional testing
  • Structural testing
  • Empirical study
  • Replication

Cite this

@article{860f284c83174f8f97a7fde82ead5db7,
title = "An Empirical Evaluation of Defect Detection Techniques",
abstract = "This report describes an empirical study comparing three defect detection techniques: (a) code reading by stepwise abstraction, (b) functional testing using equivalence partitioning and boundary value analysis, and (c) structural testing using branch coverage. It is a replication of a study that has been carried out at least four times previously over the last 20 years. This study used 47 student subjects to apply the techniques to small C programs in a fractional factorial experimental design. The major findings of the study are: (a) that the individual techniques are of broadly similar effectiveness in terms of observing failures and finding faults, (b) that the relative effectiveness of the techniques depends on the nature of the program and its faults, (c) these techniques are consistently much more effective when used in combination with each other. These results contribute to a growing body of empirical evidence that supports generally held beliefs about the effectiveness of defect detection techniques in software engineering.",
keywords = "Software testing, Code reading, Code review, Functional testing, Structural testing, Empirical study, Replication",
author = "M. Roper and M. Wood and J. Miller",
year = "1997",
doi = "10.1016/S0950-5849(97)00028-1",
language = "English",
volume = "39",
pages = "763--775",
journal = "Information and Software Technology",
issn = "0950-5849",
number = "11",

}

An Empirical Evaluation of Defect Detection Techniques. / Roper, M.; Wood, M.; Miller, J.

In: Information and Software Technology, Vol. 39, No. 11, 1997, p. 763-775.

Research output: Contribution to journalArticle

TY - JOUR

T1 - An Empirical Evaluation of Defect Detection Techniques

AU - Roper, M.

AU - Wood, M.

AU - Miller, J.

PY - 1997

Y1 - 1997

N2 - This report describes an empirical study comparing three defect detection techniques: (a) code reading by stepwise abstraction, (b) functional testing using equivalence partitioning and boundary value analysis, and (c) structural testing using branch coverage. It is a replication of a study that has been carried out at least four times previously over the last 20 years. This study used 47 student subjects to apply the techniques to small C programs in a fractional factorial experimental design. The major findings of the study are: (a) that the individual techniques are of broadly similar effectiveness in terms of observing failures and finding faults, (b) that the relative effectiveness of the techniques depends on the nature of the program and its faults, (c) these techniques are consistently much more effective when used in combination with each other. These results contribute to a growing body of empirical evidence that supports generally held beliefs about the effectiveness of defect detection techniques in software engineering.

AB - This report describes an empirical study comparing three defect detection techniques: (a) code reading by stepwise abstraction, (b) functional testing using equivalence partitioning and boundary value analysis, and (c) structural testing using branch coverage. It is a replication of a study that has been carried out at least four times previously over the last 20 years. This study used 47 student subjects to apply the techniques to small C programs in a fractional factorial experimental design. The major findings of the study are: (a) that the individual techniques are of broadly similar effectiveness in terms of observing failures and finding faults, (b) that the relative effectiveness of the techniques depends on the nature of the program and its faults, (c) these techniques are consistently much more effective when used in combination with each other. These results contribute to a growing body of empirical evidence that supports generally held beliefs about the effectiveness of defect detection techniques in software engineering.

KW - Software testing

KW - Code reading

KW - Code review

KW - Functional testing

KW - Structural testing

KW - Empirical study

KW - Replication

UR - http://dx.doi.org/doi:10.1016/S0950-5849(97)00028-1

U2 - 10.1016/S0950-5849(97)00028-1

DO - 10.1016/S0950-5849(97)00028-1

M3 - Article

VL - 39

SP - 763

EP - 775

JO - Information and Software Technology

T2 - Information and Software Technology

JF - Information and Software Technology

SN - 0950-5849

IS - 11

ER -