Tianyi Zhang Miryung Kim University of California Los Angeles 1 Problem Statement Code clones are common in modern software systems Developers often find it difficult to examine the runtime behavior of clones ID: 704837
Download Presentation The PPT/PDF document "Automated Transplantation and Differenti..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Automated Transplantation and Differential Testing for Clones
Tianyi Zhang, Miryung KimUniversity of California, Los Angeles
1Slide2
Problem Statement
Code clones are common in modern software systems.Developers often find it difficult to examine the runtime behavior of clones.
This problem is exacerbated by a lack of tests. 46% of clone pairs are only partially covered by existing test suites.
We present Grafter to reuse tests between clones and examine behavior differences.
2Slide3
public class Copy extends Task{
private IncludePatternSet includes;
public void setIncludes(String patterns){
…
StringTokenizer
tok
= new StringTokenizer(patterns, “,”); while(tok.hasMoreTokens()){ includes.addPattern(tok.next()); } … }}class IncludePatternSet { public Set<String> set; public void addPattern(String s) { set.add(s); };}
public class Delete extends Task{ private ExcludePatternSet excludes; public void setExcludes(String patterns){ … StringTokenizer tok = new StringTokenizer(patterns, “,”); while(tok.hasMoreTokens()){ excludes.addPattern(tok.next()); } … }}class ExcludePatternSet { public Set<String> set; public void addPattern(String s) { set.add(s); };}
A pair of similar but not identical clones that are detected by an existing clone detection tool, Deckard [ICSE 2007].
Copy.java
Delete.java
3
* The example is adapted from Apache Ant 1.9.6 for presentation purposes . Slide4
public class Copy extends Task{
private IncludePatternSet includes;
public void setIncludes(String patterns){
…
String[] tokens =
StringUtils.split
(patterns, “,”);
for (String tok : tokens){ includes.addPattern(tok); } … }}class IncludePatternSet { public Set<String> set; public void addPattern(String s) { set.add(s); };}public class Delete extends Task{
private ExcludePatternSet excludes; public void setExcludes(String patterns){ … String[] tokens = StringUitls.split(patterns, “.”); for (String tok : tokens) excludes.addPattern(tok); } … }}class ExcludePatternSet { public Set<String> set; public void addPattern(String s) { set.add(s); };}A programmer updates the use of StringTokenizer to StringUtils.split
in the Copy and Delete classes.
Copy.java
Delete.java4
* The example is adapted from Apache Ant 1.9.6 for presentation purposes .
test
?Slide5
Limitation of Existing Techniques
Existing test reuse technique for clones works only at a method or class level and requires a reuse plan. [Makady
& Walker]Existing differential testing or random testing techniques are not geared towards
intra method clones
[
Diffut
,
Randoop
]Existing clone inconsistency detection techniques do not detect behavioral differences between clones [Jiang et al., CBCD, SPA] 5Slide6
Grafter: Automated Test Reuse and Differential Testing
6test
Copy.java
(recipient)
public class Copy extends Task{
private
IncludePatternSet
includes;
… String[] tokens = StringUtils.split(patterns, “,”); for(String tok : tokens){ includes.addPattern
(tok); } …public class Delete extends Task{ private ExcludePatternSet excludes; … String[] tokens = StringUtils.split(patterns, “.”); for(String tok : tokens){ excludes.addPattern(tok); } …Delete.java (donor)
String[] tokens = StringUtils.split(patterns, “.”);
for(String tok
: tokens){
excludes.addPattern(
tok);
}
Success
?
FailureSlide7
7
test
Copy.java
public class Copy extends Task{
private
IncludePatternSet
includes;
…
String[] tokens = StringUtils.split(patterns, “,”); for(String tok : tokens){ includes.addPattern(tok
); } …public class Delete extends Task{ private ExcludePatternSet excludes; … String[] tokens = StringUtils.split(patterns, “.”); for(String tok : tokens){ excludes.addPattern(tok); } …Delete.javaString[] tokens = StringUtils.split(patterns, “.”); for(String
tok : tokens){
excludes.addPattern(
tok); }
Success
Failure
Grafter: Automated Test Reuse and Differential Testing
tokens
[“
src
/*.java
”, “
test/*.java
”]
tokens
[“
src
/*
”, “
java, test/*
”, “
java
”]Slide8
Grafter Approach Overview
8
Test Suite
Clone Pair
A
B
Test
Clone A
Clone B
T1
pass
fail
T2
pass
pass
State
Clone A
Clone B
S1
“string”
“string”
S2
true
falseSlide9
9
public class Delete extends Task{
private
ExcludePatternSet
excludes;
public void
setExcludes
(String patterns){
… String[] tokens = StringUtils.split(patterns, “.”); for(String
tok : tokens){ excludes.addPattern(tok); } } …public class Copy extends Task{ private IncludePatternSet includes; public void setIncludes
(String patterns){
…
String[] tokens = StringUtils.split
(patterns, “
,”);
for(String
tok
: tokens){
includes.addPattern
(
tok
);
}
}
…
Step 1: Variation IdentificationSlide10
Step 1: Variation Identification
10
public class Delete extends Task{
private
ExcludePatternSet
excludes
;
public void setExcludes(String patterns){ … String[] tokens = StringUtils.split(patterns, “.
”); for(String tok : tokens){ excludes.addPattern(tok); } } …public class Copy extends Task{ private
IncludePatternSet includes
;
public void
setIncludes
(String patterns){ …
String[] tokens =
StringUtils.split
(patterns, “
,
”);
for(String
tok
: tokens){
includes
.addPattern
(
tok
);
}
}
…Slide11
Step 2: Code Transplantation
11
public class Delete extends Task{
private
ExcludePatternSet
excludes
;
public void setExcludes(String patterns){ … String[] tokens = StringUtils.split(patterns, “.
”); for(String tok : tokens){ excludes.addPattern(tok); } } …public class Copy extends Task{ private
IncludePatternSet includes
;
public void
setIncludes
(String patterns){ …
String[] tokens =
StringUtils.split
(patterns, “
,
”);
for(String
tok
: tokens){
includes
.addPattern
(
tok
);
}
}
…
String[] tokens =
StringUtils.split
(patterns, “
.
”);
for(String
tok
: tokens){
excludes
.addPattern
(
tok
);
}Slide12
Step 2: Code Transplantation
12
public class Delete extends Task{
private
ExcludePatternSet
excludes
;
public void setExcludes(String patterns){ … String[] tokens = StringUtils.split(patterns, “.
”); for(String tok : tokens){ excludes.addPattern(tok); } } …public class Copy extends Task{ private
IncludePatternSet includes
;
+ private
ExcludePatternSet excludes;
public void
setIncludes
(String patterns){
…
+ excludes = new
ExcludePatternSet
();
String[] tokens =
StringUtils.split
(patterns, “
.
”);
for(String
tok
: tokens){
excludes
.addPattern
(
tok
);
}
}
…Slide13
Step 3: Data Propagation
13
public class Delete extends Task{
private
ExcludePatternSet
excludes
;
public void setExcludes(String patterns){ … String[] tokens = StringUtils.split(patterns, “.
”); for(String tok : tokens){ excludes.addPattern(tok); } } …public class Copy extends Task{ private
IncludePatternSet includes
;
+ private
ExcludePatternSet excludes;
public void
setIncludes
(String patterns){
…
+ excludes = new
ExcludePatternSet
();
+
excludes.set
=
includes.set
;
String[] tokens =
StringUtils.split
(patterns, “
.
”);
for(String
tok
: tokens){
excludes
.addPattern
(
tok
);
}
+
includes.set
=
excludes.set
;
}
…Slide14
Step 4: Differential Testing
14
Test
Copy.java
Delete.java
testCopy
pass
fail
State
Copy.java
Delete.java
patterns
“
src
/*.java, test/*.java”
“
src
/*.java, test/*.java”
tokens
[“
src
/*.java
”, “
test/*.java
”]
[“
src
/*
”, “
java, test/*
”, “
java
”]
in(ex)
cludes
<
IncludePatternSet
>
<set>
[“
src
/*.java
”, “
test/*.java
”
]
</set>
</
IncludePatternSet
>
<
ExcludePatternSet
>
<set>
[“
src
/*
”, “
java, test/*
”, “
java
”]
</set>
</
ExcludePatternSet
>
Test-level Comparison
State-level ComparisonSlide15
Our tool & dataset are publicly available.
15
Tool & Dataset:
http://web.cs.ucla.edu/~tianyi.zhang/grafter.html
Slide16
Behavioral differences are represented in tables and highlighted for ease of investigation.
16Slide17
Evaluation Dataset
Our dataset contains 52 pairs of non-identical clones from 3 open source projects. 38 clone pairs that have different variable names, types, method calls, etc. (Type II)14 clone pairs that have
extra statements in one clone. (Type III)
17
Subject
Version
Description
LOC
Test#Branch
StmtClone PairApache Ant1.9.6A software build framework267,048 1,86445%50%18Java APNS1.0.0A Java client for Apple Push Notification service (APNs)8,362 10359%67%7XML Security2.0.5
A XML signature and encryption library
121,594 39659%
65%27Slide18
Research Questions
RQ1. What is Grafter’s transplantation capability? RQ2. How does Grafter compare with a static approach by Jiang et al. in its ability to detect differences in clones?
RQ3. How sensitive is Grafter in detecting behavioral differences caused by mutants?
18Slide19
RQ1. Transplantation Success and Test Reuse Capability
Grafter successfully grafts 49 of 52 pairs of clonesGrafter inserts 6 lines of stub code on average to ensure type safetyGrafter doubles the test coverage for partially tested clone pairs
19Slide20
RQ2. Behavioral Difference Detection
Jiang et al. [FSE 2007] present a static approach that detects three pre-defined cloning inconsistencies: (1) renaming mistake, (2) control construct inconsistency, and (3)
control predicate inconsistency
Grafter exposes behavioral differences in 84% clone pairs while Jiang et al. detect syntactic inconsistency in 33% clone pairs only.
20
Grafter
Jiang et al.Slide21
We inject 361 mutants using Major [ISSTA 2014]. Hypothesis: Grafter should be effective to detect behavior differences induced by mutants.
RQ3. Robustness
21
public void
setType
(String
type
) {
if ( type == null && type.length() == 0){ this.type = null; } else { URI tmpType = null; try { tmpType = new URI(type); } catch (URISyntaxException ex) { … } this.type = tmpType.toString
(); }}Mutation Example from Apache XML Securitypublic void setEncoding(String encoding) { if (encoding == null && encoding.length() == 0){ this.encoding = null; } else {
URI tmpEncoding = null; try {
tmpEncoding = new URI(encoding
); } catch (URISyntaxException ex) {
… }
this.encoding = tmpEncoding.toString
();
}
}
type
!= null Slide22
RQ3. Robustness
Grafter detects 36% more mutants using the test-level comparison and almost 2X more mutants using the state-level comparisonGrafter is less biased to mutant types than Jiang et al.
22Slide23
Conclusion
This work introduces the first test transplantation and reuse approach for enabling runtime behavior comparison between clones.Grafter’s code transplantation succeeds in 94% of the cases
。
The
fine
-
grained differential testing can detect up to 2X more seeded faults than a baseline static cloning bug finder.
23Slide24
Q&A24Slide25
Why not transplant test cases?25
method A
method B
tes
t1
m1
m2
m3
Modify test1 to call m3?
Intra-method clones cannot be easily invoked directly from a test.
It’s not easy to reconstruct the calling context of code clones under test.Slide26
Why not transplant test cases?26
method A
method B
tes
t1
m1
m2
m3
Intra-method clones cannot be directly invoked from a test.
It’s not easy to match the call stack of code clones under test.
TransplantSlide27
Are these behavior differences real bugs?Code clones may have intended behavioral differences. [Kim et al. FSE 2015]
We do not assume that syntactically similar code must behave the same nor we argue that Grafter is a bug finding tool.Grafter rather enables developers to investigate behavioral differences between code clones through fine-grained differential testing.
27Slide28
Clones may have variations in variable names, types, method call targets, etc. (Type II clones
)28
If (cert
instanceof
X509Certificate) {
XMLX509SubjectName
certSN = new XMLX509SubjectName( x509SubjectName.getDocument(), (X509Certificate)cert); …If (cert instanceof X509Certificate) { XMLX509IssuerSerial certSerial = new XMLX509IssuerSerial( x509Serial.getDocument(), (X509Certificate)cert); …
Clones may include additional code insertions and deletions. (Type III clones)if (f.isDirectory()) { deleteDir(f);} else { log(“Deleting ” + f.getPath()); if (!delete(f)) { handle(“Unable to delete file” + …); }}if (f.isDirectory()) { deleteDir
(f);} else if (deleteFiles
&& ! getFileUtils.tryHardToDelete) {
throw new BuildException (“Unable to delete file” …);
} else { throw new
BuildException (“Unexpected Error. File should not exist!”);}
Apache XML Security
Apache AntSlide29
Why not use existing code transplantation techniques? Tool Name [Barr et al., 2015] transplants code so it can pass test cases in the target location for the purpose of repair.
Our grafting is not guided by test oracles but leverages syntactic resemblance between clones, e.g., mapping between variables and methods.
29Slide30
Why use Jiang et al. as a baseline?Jiang et al. is a static approach that compares control and data-flow dependence differences to find potential inconsistencies between clones. We compare Grafter against Jiang et al. to investigate how our differential testing approach is sensitive to detect behavior differences than a static approach.
30Slide31
Jiang et al. detect three types of syntactic inconsistencies between clones.
31
Jiang et al.’s Clone Inconsistency Detection
for (
i
= 0;
i
<
response_count; i++) { msgbuf[i*3] = 0; msg(DBG_SQ1, “cc_ReadSubQ: %s\n”, msgbuf);}if (cmd_type == READ_M2){ msgbuf[xa_count*3] = 0; msg(DBG_XA1, “xa head: %s\n”,
msgbuf);}if (length >=11 && strcmp (buffer, “CUMANNASCSI2”) == 0) { buffer += 9; length += 9;… }if (length >=9 && strncmp (buffer, “EESOXSCSI”, 9) == 0) { buffer += 9; length += 9;… }if (l_stride != NULL) { mps_cdiv_q (X1, X1, r_stride->value);if (l_stride != NULL) { mps_cdiv_q (X1, X1, l_stride->value);
Example 1. Control Construct Inconsistency
Example 2. Control Predicate Inconsistency
Example 3. Renaming Mistake