Tung Dao Lingming Zhang Na Meng Virginia Tech The University of Texas at Dallas Background Bug localization is important in software maintenance Bug localization is challenging and timeconsuming ID: 645286
Download Presentation The PPT/PDF document "How Does Execution Information Help with..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
How Does Execution Information Help with Information-Retrieval Based Bug Localization?
Tung Dao*
Lingming
Zhang
+
Na Meng*
Virginia Tech*
The University of Texas at DallasSlide2
Background
Bug localization is important in software maintenance
Bug localization is challenging and time-consuming
2Slide3
Information Retrieval (IR)-Based Bug Localization
Static information
Buggy Program
Indexing
Bug Report
Query Construction
Word-indexed Software Entities (i.e., classes and methods)
query
Ranked list of software entities
Ranking
3Slide4
Spectrum-Based Bug Localization
Dynamic information
Buggy Program
Test Cases
Program Execution
Execution information of each passed or failed test
Suspiciousness Calculation
Ranked list of suspicious bug locations
4Slide5
5
By combining static and dynamic information, can we improve bug localization?Slide6
Contributions
We systematically investigated ways to combine static and dynamic information
Three IR-based techniques
B
aseline,
BugLocator
[1], and
BLUiR[2]Three kinds of execution informationCoverage, slicing, and spectrumF
our ways to calculate spectrum (Tarantula[3], Ochiai[4], Jaccard[5], and Ample[6]) Four ways of combining information
6Slide7
Three Types of Execution Information
With
method-level instrumentation, we collected
Coverage
We recorded
classes or methods executed by the failed test(s)
Slicing
We recorded executed classes or methods relevant to a failureSpectrum
We calculated the suspiciousness of each executed class or method7Slide8
Two Strategies
Search space reduction
The coverage or slicing information of failed tests can refine IR-based bug localization results
Rank tuning
The ranked list of spectrum information can improve the ranking of IR-based bug localization results
8Slide9
Search Space Reduction
9
R = N - M
IR-based bug localization result
Locations not covered by any failed test
Refined IR-based ranked list
IR
cov
and
IR
sliceSlide10
Rank Tuning
10
IR-based code location rank list
Spectrum-based code location rank list
List Merger
Tuned rank list
IR
cov
-spec
and
IR
slice
-specSlide11
Experiment Data
Existing benchmark suite published by Le et al. [7]
11Slide12
12
IR vs.
IR
cov
Metric
Project
(a) Class Level
Baseline
BugLocator
BLUiR
IR
IR
cov
IR
IR
cov
IR
IR
cov
Top 1
AspectJ
4
6
2
6
3
5
Ant
27
32
22
33
26
31
Lucene
12
14
6
15
11
14
Rhino
4
4
5
9
11
11
Overall
47
56
35
63
51
61
Metric
Project
(a) Method Level
Baseline
BugLocator
BLUiR
IR
IR
cov
IR
IR
cov
IR
IR
cov
Top 1
AspectJ
3
3
2
4
2
3
Ant
9
12
10
13
6
13
Lucene473957Rhino464656Overall202819321829
At class level, coverage consistently improved all studied IR-based techniques
At method level, coverage improved IR-based bug localization more significantly than at class-levelSlide13
13
IR
cov
vs.
IR
slice
Metric
Project
(a) Class Level
Baseline
BugLocator
BLUiR
IR
cov
IR
slice
IR
IR
covIR
IRcov
MAP
Ant
0.71
0.79
0.71
0.75
0.71
0.85
Lucene
0.50
0.58
0.44
0.62
0.44
0.54
Rhino
0.47
0.50
0.50
0.50
0.48
0.58
Overall
0.56
0.62
0.55
0.62
0.54
0.66
Metric
Project
(a) Method Level
Baseline
BugLocator
BLUiR
IR
IR
cov
IR
IR
cov
IR
IR
cov
MAP
Ant
0.37
0.52
0.33
0.34
0.35
0.52
Lucene
0.30
0.45
0.35
0.45
0.34
0.46
Rhino
0.02
0.04
0.02
0.11
0.020.04Overall0.230.340.230.300.240.34Slicing was more helpful than coverage in improving IR-based techniquesSlide14
IRcov vs.
IR
cov
-spec
14
Spectrum information was effective to improve
IR
cov
at method level instead of at class levelWith Tarantula and α=0.7, IRcov-spec almost always achieved the best performanceSlide15
IRslice vs.
IR
slice
-spec
Spectrum information was effective to improve
IR
slice
at method level instead of at class level
With Tarantula and α=0.7,
IR
slice
-spec
almost always achieved the best performance15Slide16
Comparison with AML[7]
16
IR
cov
outperformed AML in all metrics but one
IR
cov
-spec
outperformed AML
Metric
Project
IR
IRcov
IRcov
-spec
AML
Top 5
AspectJ4714
13
Ant
16
36
39
22
Lucene
16
32
23
22
Rhino
7
15
22
14
Overall
43
90
98
71Slide17
Conclusion
Various execution information can effectively help with IR-based bug localization
With search space reduction strategy, we can remove irrelevant code locations from the IR-based lists, and improve the ranking of actual buggy locations
With rank tuning, we can improve the ranking of actual buggy locations that are covered by many failed tests
17Slide18
Conclusion
We need to investigate more approaches to handle some hard-to-find bugs
Investigated techniques still do not work well when bug location information is not mentioned in the bug report
When a lot of code location information is provided in the bug report, current approaches cannot always effectively identify the bug location
18Slide19
References
[1] J
. Zhou, H. Zhang, and D. Lo. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports. In
Software Engineering (ICSE), 2012 34th International Conference on
, pages 14–24, June 2012.
[2] R
.
Saha, M. Lease, S. Khurshid, and D. Perry. Improving bug local- ization using structured information retrieval. In
Automated Software Engineering (ASE), 2013 IEEE/ACM 28th International Conference on, pages 345–355, Nov 2013. [3] J. A. Jones, M. J. Harrold, and J. Stasko. Visualization of test information to assist fault localization. In Proceedings of the 24th International Conference on Software Engineering, 2002.
[4] R. Abreu, P. Zoeteweij, and A. J. C. v. Gemund. An evaluation of similarity coefficients for software fault localization. In
Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing, PRDC ’06, pages 39–46, Washington, DC, USA, 2006. IEEE Computer Society. 19Slide20
References
[5]
R. Abreu, P.
Zoeteweij, and A. van
Gemund
. On the accuracy of spectrum-based fault localization. In
Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION, 2007. TAICPART-MUTATION 2007
, pages 89–98, Sept 2007. [6] V. Dallmeier, C. Lindig
, and A. Zeller. Lightweight bug localization with ample. In Proceedings of the Sixth International Symposium on Automated Analysis-driven Debugging, 2005. [7] T.-D. B. Le, R. J. Oentaryo, and D. Lo. Information retrieval and spectrum based bug localization: Better together. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering
, ESEC/FSE 2015, pages 579–590, New York, NY, USA, 2015. ACM
20