/
How Does Execution Information Help with Information-Retrieval Based Bug Localization? How Does Execution Information Help with Information-Retrieval Based Bug Localization?

How Does Execution Information Help with Information-Retrieval Based Bug Localization? - PowerPoint Presentation

test
test . @test
Follow
366 views
Uploaded On 2018-03-10

How Does Execution Information Help with Information-Retrieval Based Bug Localization? - PPT Presentation

Tung Dao Lingming Zhang Na Meng Virginia Tech The University of Texas at Dallas Background Bug localization is important in software maintenance Bug localization is challenging and timeconsuming ID: 645286

information bug cov based bug information based cov localization level spectrum software list spec slice class rank improve method

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "How Does Execution Information Help with..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

How Does Execution Information Help with Information-Retrieval Based Bug Localization?

Tung Dao*

Lingming

Zhang

+

Na Meng*

Virginia Tech*

The University of Texas at DallasSlide2

Background

Bug localization is important in software maintenance

Bug localization is challenging and time-consuming

2Slide3

Information Retrieval (IR)-Based Bug Localization

Static information

Buggy Program

Indexing

Bug Report

Query Construction

Word-indexed Software Entities (i.e., classes and methods)

query

Ranked list of software entities

Ranking

3Slide4

Spectrum-Based Bug Localization

Dynamic information

Buggy Program

Test Cases

Program Execution

Execution information of each passed or failed test

Suspiciousness Calculation

Ranked list of suspicious bug locations

4Slide5

5

By combining static and dynamic information, can we improve bug localization?Slide6

Contributions

We systematically investigated ways to combine static and dynamic information

Three IR-based techniques

B

aseline,

BugLocator

[1], and

BLUiR[2]Three kinds of execution informationCoverage, slicing, and spectrumF

our ways to calculate spectrum (Tarantula[3], Ochiai[4], Jaccard[5], and Ample[6]) Four ways of combining information

6Slide7

Three Types of Execution Information

With

method-level instrumentation, we collected

Coverage

We recorded

classes or methods executed by the failed test(s)

Slicing

We recorded executed classes or methods relevant to a failureSpectrum

We calculated the suspiciousness of each executed class or method7Slide8

Two Strategies

Search space reduction

The coverage or slicing information of failed tests can refine IR-based bug localization results

Rank tuning

The ranked list of spectrum information can improve the ranking of IR-based bug localization results

8Slide9

Search Space Reduction

9

R = N - M

IR-based bug localization result

Locations not covered by any failed test

Refined IR-based ranked list

IR

cov

and

IR

sliceSlide10

Rank Tuning

10

IR-based code location rank list

Spectrum-based code location rank list

List Merger

Tuned rank list

IR

cov

-spec

and

IR

slice

-specSlide11

Experiment Data

Existing benchmark suite published by Le et al. [7]

11Slide12

12

IR vs.

IR

cov

Metric

Project

(a) Class Level

Baseline

BugLocator

BLUiR

IR

IR

cov

IR

IR

cov

IR

IR

cov

Top 1

AspectJ

4

6

2

6

3

5

Ant

27

32

22

33

26

31

Lucene

12

14

6

15

11

14

Rhino

4

4

5

9

11

11

Overall

47

56

35

63

51

61

Metric

Project

(a) Method Level

Baseline

BugLocator

BLUiR

IR

IR

cov

IR

IR

cov

IR

IR

cov

Top 1

AspectJ

3

3

2

4

2

3

Ant

9

12

10

13

6

13

Lucene473957Rhino464656Overall202819321829

At class level, coverage consistently improved all studied IR-based techniques

At method level, coverage improved IR-based bug localization more significantly than at class-levelSlide13

13

IR

cov

vs.

IR

slice

Metric

Project

(a) Class Level

Baseline

BugLocator

BLUiR

IR

cov

IR

slice

IR

IR

covIR

IRcov

MAP

Ant

0.71

0.79

0.71

0.75

0.71

0.85

Lucene

0.50

0.58

0.44

0.62

0.44

0.54

Rhino

0.47

0.50

0.50

0.50

0.48

0.58

Overall

0.56

0.62

0.55

0.62

0.54

0.66

Metric

Project

(a) Method Level

Baseline

BugLocator

BLUiR

IR

IR

cov

IR

IR

cov

IR

IR

cov

MAP

Ant

0.37

0.52

0.33

0.34

0.35

0.52

Lucene

0.30

0.45

0.35

0.45

0.34

0.46

Rhino

0.02

0.04

0.02

0.11

0.020.04Overall0.230.340.230.300.240.34Slicing was more helpful than coverage in improving IR-based techniquesSlide14

IRcov vs.

IR

cov

-spec

14

Spectrum information was effective to improve

IR

cov

at method level instead of at class levelWith Tarantula and α=0.7, IRcov-spec almost always achieved the best performanceSlide15

IRslice vs.

IR

slice

-spec

Spectrum information was effective to improve

IR

slice

at method level instead of at class level

With Tarantula and α=0.7,

IR

slice

-spec

almost always achieved the best performance15Slide16

Comparison with AML[7]

16

IR

cov

outperformed AML in all metrics but one

IR

cov

-spec

outperformed AML

Metric

Project

IR

IRcov

IRcov

-spec

AML

Top 5

AspectJ4714

13

Ant

16

36

39

22

Lucene

16

32

23

22

Rhino

7

15

22

14

Overall

43

90

98

71Slide17

Conclusion

Various execution information can effectively help with IR-based bug localization

With search space reduction strategy, we can remove irrelevant code locations from the IR-based lists, and improve the ranking of actual buggy locations

With rank tuning, we can improve the ranking of actual buggy locations that are covered by many failed tests

17Slide18

Conclusion

We need to investigate more approaches to handle some hard-to-find bugs

Investigated techniques still do not work well when bug location information is not mentioned in the bug report

When a lot of code location information is provided in the bug report, current approaches cannot always effectively identify the bug location

18Slide19

References

[1] J

. Zhou, H. Zhang, and D. Lo. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports. In

Software Engineering (ICSE), 2012 34th International Conference on

, pages 14–24, June 2012.

[2] R

.

Saha, M. Lease, S. Khurshid, and D. Perry. Improving bug local- ization using structured information retrieval. In

Automated Software Engineering (ASE), 2013 IEEE/ACM 28th International Conference on, pages 345–355, Nov 2013. [3] J. A. Jones, M. J. Harrold, and J. Stasko. Visualization of test information to assist fault localization. In Proceedings of the 24th International Conference on Software Engineering, 2002.

[4] R. Abreu, P. Zoeteweij, and A. J. C. v. Gemund. An evaluation of similarity coefficients for software fault localization. In

Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing, PRDC ’06, pages 39–46, Washington, DC, USA, 2006. IEEE Computer Society. 19Slide20

References

[5]

R. Abreu, P.

Zoeteweij, and A. van

Gemund

. On the accuracy of spectrum-based fault localization. In

Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION, 2007. TAICPART-MUTATION 2007

, pages 89–98, Sept 2007. [6] V. Dallmeier, C. Lindig

, and A. Zeller. Lightweight bug localization with ample. In Proceedings of the Sixth International Symposium on Automated Analysis-driven Debugging, 2005. [7] T.-D. B. Le, R. J. Oentaryo, and D. Lo. Information retrieval and spectrum based bug localization: Better together. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

, ESEC/FSE 2015, pages 579–590, New York, NY, USA, 2015. ACM

20