A preliminary evaluation of text-based and dependency-based techniques for determining the origins of bugs

Steven Davies, Marc Roper, Murray Wood

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

2 Citations (Scopus)

Abstract

A crucial step in understanding the life cycle of software bugs is identifying their origin. Unfortunately this information is not usually recorded and recovering it at a later date is challenging. Recently two approaches have been developed that attempt to solve this problem: the text approach and the dependency approach. However only limited evaluation has been carried out on their effectiveness so far, partially due to the lack of data sets linking bugs to their introduction. Producing such data sets is both time-consuming and challenging due to the subjective nature of the problem. To improve this, the origins of 166 bugs in two open-source projects were manually identified. These were then compared to a simulation of the approaches. The results show that both approaches were partially successful across a variety of different types of bugs. They achieved a precision of 29%{79% and a recall of 40%{70%, and could perform better when combined. However there remain a number of challenges to overcome in future development|large commits, unrelated changes and large numbers of versions between the origin and the x all reduce their effectiveness.
LanguageEnglish
Title of host publication18th Working Conference on Reverse Engineering (WCRE 2011)
PublisherIEEE
Pages201-210
Number of pages10
ISBN (Print)978-1-4577-1948-6
DOIs
Publication statusPublished - Oct 2011
Event18th Working Conference on Reverse Engineering (WCRE), 2011 - Limerick, Ireland
Duration: 17 Oct 201120 Oct 2011

Conference

Conference18th Working Conference on Reverse Engineering (WCRE), 2011
CountryIreland
CityLimerick
Period17/10/1120/10/11

Fingerprint

Life cycle

Keywords

  • software maintenance
  • bug-introducing changes
  • mining software repositories

Cite this

Davies, Steven ; Roper, Marc ; Wood, Murray. / A preliminary evaluation of text-based and dependency-based techniques for determining the origins of bugs. 18th Working Conference on Reverse Engineering (WCRE 2011). IEEE, 2011. pp. 201-210
@inproceedings{f118d4c2d40e40fcad5fbb625a4a03db,
title = "A preliminary evaluation of text-based and dependency-based techniques for determining the origins of bugs",
abstract = "A crucial step in understanding the life cycle of software bugs is identifying their origin. Unfortunately this information is not usually recorded and recovering it at a later date is challenging. Recently two approaches have been developed that attempt to solve this problem: the text approach and the dependency approach. However only limited evaluation has been carried out on their effectiveness so far, partially due to the lack of data sets linking bugs to their introduction. Producing such data sets is both time-consuming and challenging due to the subjective nature of the problem. To improve this, the origins of 166 bugs in two open-source projects were manually identified. These were then compared to a simulation of the approaches. The results show that both approaches were partially successful across a variety of different types of bugs. They achieved a precision of 29{\%}{79{\%} and a recall of 40{\%}{70{\%}, and could perform better when combined. However there remain a number of challenges to overcome in future development|large commits, unrelated changes and large numbers of versions between the origin and the x all reduce their effectiveness.",
keywords = "software maintenance, bug-introducing changes, mining software repositories",
author = "Steven Davies and Marc Roper and Murray Wood",
year = "2011",
month = "10",
doi = "10.1109/WCRE.2011.32",
language = "English",
isbn = "978-1-4577-1948-6",
pages = "201--210",
booktitle = "18th Working Conference on Reverse Engineering (WCRE 2011)",
publisher = "IEEE",

}

Davies, S, Roper, M & Wood, M 2011, A preliminary evaluation of text-based and dependency-based techniques for determining the origins of bugs. in 18th Working Conference on Reverse Engineering (WCRE 2011). IEEE, pp. 201-210, 18th Working Conference on Reverse Engineering (WCRE), 2011, Limerick, Ireland, 17/10/11. https://doi.org/10.1109/WCRE.2011.32

A preliminary evaluation of text-based and dependency-based techniques for determining the origins of bugs. / Davies, Steven; Roper, Marc; Wood, Murray.

18th Working Conference on Reverse Engineering (WCRE 2011). IEEE, 2011. p. 201-210.

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - A preliminary evaluation of text-based and dependency-based techniques for determining the origins of bugs

AU - Davies, Steven

AU - Roper, Marc

AU - Wood, Murray

PY - 2011/10

Y1 - 2011/10

N2 - A crucial step in understanding the life cycle of software bugs is identifying their origin. Unfortunately this information is not usually recorded and recovering it at a later date is challenging. Recently two approaches have been developed that attempt to solve this problem: the text approach and the dependency approach. However only limited evaluation has been carried out on their effectiveness so far, partially due to the lack of data sets linking bugs to their introduction. Producing such data sets is both time-consuming and challenging due to the subjective nature of the problem. To improve this, the origins of 166 bugs in two open-source projects were manually identified. These were then compared to a simulation of the approaches. The results show that both approaches were partially successful across a variety of different types of bugs. They achieved a precision of 29%{79% and a recall of 40%{70%, and could perform better when combined. However there remain a number of challenges to overcome in future development|large commits, unrelated changes and large numbers of versions between the origin and the x all reduce their effectiveness.

AB - A crucial step in understanding the life cycle of software bugs is identifying their origin. Unfortunately this information is not usually recorded and recovering it at a later date is challenging. Recently two approaches have been developed that attempt to solve this problem: the text approach and the dependency approach. However only limited evaluation has been carried out on their effectiveness so far, partially due to the lack of data sets linking bugs to their introduction. Producing such data sets is both time-consuming and challenging due to the subjective nature of the problem. To improve this, the origins of 166 bugs in two open-source projects were manually identified. These were then compared to a simulation of the approaches. The results show that both approaches were partially successful across a variety of different types of bugs. They achieved a precision of 29%{79% and a recall of 40%{70%, and could perform better when combined. However there remain a number of challenges to overcome in future development|large commits, unrelated changes and large numbers of versions between the origin and the x all reduce their effectiveness.

KW - software maintenance

KW - bug-introducing changes

KW - mining software repositories

UR - http://www.scopus.com/inward/record.url?scp=83455162490&partnerID=8YFLogxK

UR - http://www.cs.wm.edu/semeru/wcre2011/

U2 - 10.1109/WCRE.2011.32

DO - 10.1109/WCRE.2011.32

M3 - Conference contribution book

SN - 978-1-4577-1948-6

SP - 201

EP - 210

BT - 18th Working Conference on Reverse Engineering (WCRE 2011)

PB - IEEE

ER -