Repair Shauvik Roy Choudhary Dan Zhao Husayn Versee and Alessandro Orso Georgia Institute of Technology Web Application Testing 2 Test Machine Web Server Test Slave httpwwwwebsitecom ID: 782681
Download The PPT/PDF document "WATER Web Application TEst" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
WATERWeb Application TEst Repair
Shauvik Roy Choudhary,
Dan Zhao,
Husayn
Versee
, and Alessandro
Orso
Georgia Institute of Technology
Slide2Web Application Testing2
Test Machine
Web Server
Test Slave
http://www.website.com
Web
Application
Tests
Browser
Automation Framework
Web Application
Web
Application
Tests
Web
Application
Tests
Slide3Web Application Test Repair3
Test Machine
Web Server
Test Slave
http://www.website.com
Web
Application
Tests
Browser
Automation Framework
Web Application*
Changed
Outdated tests
Web
Application
Tests*
Web
Application
Tests*
Web
Application
Tests*
Slide4Motivating Example
<
html
>
<
body>
<h2>My Web Search
</h2>
<form
> <
input type="text" name="q"
/> <input
id="search" type="submit" value="Search"></form
><div>
<a href="http://www.gatech.edu/">
Georgia Institute of Technology </
a> <
p>Includes offices, departments, news room, professional education...
</p> <span
>www.gatech.edu</span>
</div><div>
...</div><div
>...</div>
</body></
html>
4
HTML Source of
“My Web Search”
Slide5Motivating Example
<
html
>
<
body>
<h2>My Web Search
</h2>
<form
> <
input type="text" name="q"
/> <input
id="search" type="submit" value="Search"></form
><div>
<a href="http://www.gatech.edu/">
Georgia Institute of Technology </
a> <
p>Includes offices, departments, news room, professional education...
</p> <span
>www.gatech.edu</span>
</div><div>
...</div><div
>...</div>
</body></
html>
5
Command
Arg 1Arg 2open
/search.php
type
q
Georgia Tech
clickAndWait
id=search
assertText
document.links
[0]
Georgia Institute of Technology
assertText
//body/div[1]/span
www.gatech.edu
A Selenium Test Case for “My Web Search”
HTML Source of
“My Web Search”
Slide6Motivating Example
<
html
>
<
body>
<h2>My Web Search
</h2>
<form
> <
input type="text" name="q"
/> <
input id="
searchBtn" type="submit" value="Search"></form
><div
> <
a href="http://www.gatech.edu/"> Georgia Institute of Technology
</a>
<p>Includes offices, departments, news room, professional education...
</p> <span
>www.gatech.edu</span></
div><div>
...</div><
div>...</div
></
body></
html>
6
Command
Arg
1
Arg
2
open
/search.php
type
q
Georgia Tech
clickAndWait
id=search
assertText
document.links
[0]
Georgia Institute of Technology
assertText
//body/div[1]/span
www.gatech.edu
HTML Source of
“My Web Search”
A Selenium Test Case for “My Web Search”
Change 1
Slide7Motivating Example
<
html
>
<
body>
<h2>My Web Search
</h2>
<form
> <
input type="text" name="q"
/> <input
id="search" type="submit" value="Search"> <
/form><div
> <a
href="http://www.gatech.edu/"> Georgia
Tech </a
> <p
>Includes offices, departments, news room, professional education...</p>
<span>www.gatech.edu</
span></div>
<div>...</div
><div>...
</div></
body>
</html>
7
CommandArg 1
Arg
2
open
/search.php
type
q
Georgia Tech
clickAndWait
id=search
assertText
document.links
[0]
Georgia Institute of Technology
assertText
//body/div[1]/span
www.gatech.edu
HTML Source of
“My Web Search”
A Selenium Test Case for “My Web Search”
Change 2
Slide8Motivating Example
<
html
>
<
body>
<h2>My Web Search
</h2>
<form
> <
input type="text" name="q"
/> <input
id="search" type="submit" value="Search"> <a href="adv.php">Advanced Search</a>
</form><
div>
<a href="http://www.gatech.edu/"> Georgia Institute of Technology
</a
> <p
>Includes offices, departments, news room, professional education...</p>
<span>www.gatech.edu</
span></div>
<div>...</div
><div>...
</div></
body>
</html>
8
CommandArg 1
Arg
2
open
/search.php
type
q
Georgia Tech
clickAndWait
id=search
assertText
document.links
[0]
Georgia Institute of Technology
assertText
//body/div[1]/span
www.gatech.edu
HTML Source of
“My Web Search”
A Selenium Test Case for “My Web Search”
Change
3
Slide9Motivating Example
<
html
>
<
body>
<h2>My Web Search
</h2>
<form
> <
input type="text" name="q"
/> <input
id="search" type="submit" value="Search"></form
><div id=“container”>
<div>
<a
href="http://www.gatech.edu/"> Georgia Institute of Technology
</a> <
p>Includes offices, departments, news room, professional education...</
p> <span>
www.gatech.edu</span></div
><div>...
</div><div
>...</div>
</div>
</body>
</html>
9
Command
Arg
1
Arg
2
open
/search.php
type
q
Georgia Tech
clickAndWait
id=search
assertText
document.links
[0]
Georgia Institute of Technology
assertText
//body/div[1]/span
www.gatech.edu
HTML Source of
“My Web Search”
A Selenium Test Case for “My Web Search”
Change
4
Slide10Types of ChangesStructural Changes
leads to
locator errors
form-data population problem
Content Changes
leads toobsolete content problem
Blind Changesdo not reflect on the client side 10
Slide11Technique11
Test
Web-App
old
Web-App
new
Execute test
DOM Tree
DOM Tree
+
Failure Message
(1)
Tests
with
repairs
Tests
with
repairs
Tests
with
repairs
(2)
Generate
repair candidates
Discard failed
Suggest passed
Execute candidates
(3)
Slide12Repairing LocatorsError messages:“Locator not found” – Moved Element
“Assertion failed” – Wrongly selected element
12
Old DOM Tree
body
div
div
div
h
1
a
ul
div
div
div
ul
div
div
New DOM Tree
body
div
div
div
h
1
a
ul
div
div
div
ul
div
div
div
div
div
div
div
div
div
div
DOM Node locating properties = { id,
xpath
, class,
linkText
, name }
Slide13Repairing Asserts & Form dataRepairing failing assertsReplace expected value with actual valueNegate assertion condition
Form data population
Find newly added elements in the DOM
Choose random values for the elements to generate repair candidates
13
Slide14Empirical EvaluationRQ1: Can WATER suggest repairs for most broken test scripts in a web application?
RQ2:
Can WATER do so without providing too many false positives?
14
Slide15SubjectsThree real world web applications with test cases.
15
Type
Content
Management System
Business
Process Management
User Automation Scripts
Slide16SubjectsThree real world web applications with test cases.
16
Type
Content
Management System
Business
Process Management
User Automation Scripts
Platform
PHP
Java
DSL
Slide17SubjectsThree real world web applications with test cases.
17
Type
Content
Management System
Business
Process Management
User Automation Scripts
Platform
PHP
Java
DSL
Test Scripts42 selenium tests16 selenium tests
180 (out of 5123) scriptstranslated to selenium
Slide18SubjectsThree real world web applications with test cases.
18
Type
Content
Management System
Business
Process Management
User Automation Scripts
Platform
PHP
Java
DSL
Test Scripts42 selenium tests16 selenium tests
180 (out of 5123) scriptstranslated to seleniumTest scripts run onVersions from SVN
Versions from SVN
Live site daily
Slide19Results
Versions
T
broken
C
fail
SavgTfix
Vfix
19478 – 19480
2
(4,4)
(3,3)
2
1948420430 – 20431
1(1)
(1)120448
20739 – 207401
(1)(0)0
20776
20769 – 207702
(1,1)(1,1)2
2077719
Versions
TbrokenCfailS
avgTfix
Vfix
963271 – 9634102
(1,1)
(1,1)
1
v1.7
997469 – 997470
1
(5)
(1.4)
1
v1.29
Slide20Results (continued)
20
Script
Domain
Changed
Fixed
Suggestions
10043
careers.yahoo.com
FormNo
(15, 15, 0)
10754
www.icade.fr
Form, Reloc.No0
11525
www.terra.com.brLink textYes
285
13389www.terra.com.br
Link textYes
28518164
www.google.com.brLink text
Yes1
Slide21DiscussionRQ1: EffectivenessWATER could suggest repairs to broken test casesJoomla
&
Ofbiz
studies: Suggestions correspond to actual fixes made by developers.
RQ2: Efficiency (wrong suggestions)
Joomla & Ofbiz study: 1-3 suggestionsCoscripter study: 1-285 suggestions
21
Slide22Related WorkRepair of JUnit testsDaniel et. al (
ReAssert
)
[ASE’09][ISSTA’10]
Regression testing for Web applications
Alshahwan & Harman [ICST’08] Dobolyi & Weimer [ISSRE’09]Roest,
Mesbah & Van Deursen [ICST’10]GUI Test repairMemon
& Soffa [FSE’03]Grechanik, Xie
& Fu [ICSE’09]22
Slide23Summary23