Concolic Testing of Smartphone Apps Saswat Anand Stanford Univ Mayur Naik Georgia Tech Hongseok Yang Univ of Oxford Mary Jean Harrold Georgia Tech Motivation ID: 272969
Download Presentation The PPT/PDF document "Automated" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Automated Concolic Testing of Smartphone Apps
Saswat
Anand
Stanford Univ
.
Mayur
NaikGeorgia Tech.
Hongseok YangUniv. of Oxford
Mary Jean
Harrold
Georgia Tech.Slide2
MotivationSlide3
MotivationProblems with Smartphone AppsSlide4
Automatically generate test inputs for b
ounded exhaustive
testing
of smartphone apps
ProblemSlide5
Test Inputs for AppsWhole-program testingTest input is a sequence of events e
1, e2
…, en
Types of events: a tap on the screen,
change in geo-location, arrival of a SMS message, etc.Slide6
Bounded Exhaustive Testing of AppsS, the set of all event sequences*
s.t. each sequence takes a unique path
*of bounded-length
Set of covered branches
Goal: cover theseSlide7
Generate individual eventsGenerate sequences of eventsTwo subproblemsSlide8
Generating Individual EventsAn event is associated with d
ataX & Y coordinates of a tap eventgeo-location of a change-in-geo-location event
content of an incoming SMS evente
tc.Data determine which program path is taken
Challenge: Generate the “right” data for eventsSlide9
Example: Music Player AppPlay
Pause
Stop
Eject
Rewind
SkipSlide10
Example: Music Player Apppublic
void onClick(View target) {
if (target == play)
startService(new Intent(ACTION_PLAY));
else
if (target == pause)
startService(new Intent(ACTION_PAUSE));
else if (target == skip) startService(new
Intent(ACTION_SKIP)); else if (target == rewind)
startService(new Intent(ACTION_REWIND));
else
if (target == stop)
startService(new
Intent(ACTION_STOP));
else
if (target == eject)
showUrlDialog
();
}
tap(136, 351)Slide11
Example: Music Player Apppublic
void onClick(View target) {
if (target == play)
startService(new Intent(ACTION_PLAY));
else
if (target == pause)
startService(new Intent(ACTION_PAUSE));
else if (target == skip) startService(new
Intent(ACTION_SKIP)); else if (target == rewind)
startService(new Intent(ACTION_REWIND));
else
if (target == stop)
startService(new
Intent(ACTION_STOP));
else
if (target == eject)
showUrlDialog
();
}
tap(248, 351)Slide12
Example: Music Player Apppublic
void onClick(View target) {
if (target == play)
startService(new Intent(ACTION_PLAY));
else
if (target == pause)
startService(new Intent(ACTION_PAUSE));
else if (target == skip) startService(new
Intent(ACTION_SKIP)); else if (target == rewind)
startService(new Intent(ACTION_REWIND));
else
if (target == stop)
startService(new
Intent(ACTION_STOP));
else
if (target == eject)
showUrlDialog
();
}
tap(360,
351)Slide13
Example: Music Player Apppublic
void onClick(View target) {
if (target == play)
startService(new Intent(ACTION_PLAY));
else
if (target == pause)
startService(new Intent(ACTION_PAUSE));
else if (target == skip) startService(new
Intent(ACTION_SKIP)); else if (target == rewind)
startService(new Intent(ACTION_REWIND));
else
if (target == stop)
startService(new
Intent(ACTION_STOP));
else
if (target == eject)
showUrlDialog
();
}
tap(24,
351)Slide14
Example: Music Player Apppublic
void onClick(View target) {
if (target == play)
startService(new Intent(ACTION_PLAY));
else
if (target == pause)
startService(new Intent(ACTION_PAUSE));
else if (target == skip) startService(new
Intent(ACTION_SKIP)); else if (target == rewind)
startService(new Intent(ACTION_REWIND));
else
if (target == stop)
startService(new
Intent(ACTION_STOP));
else
if (target == eject)
showUrlDialog
();
}
tap(136, 493)Slide15
Example: Music Player Apppublic
void onClick(View target) {
if (target == play)
startService(new Intent(ACTION_PLAY));
else
if (target == pause)
startService(new Intent(ACTION_PAUSE));
else if (target == skip) startService(new
Intent(ACTION_SKIP)); else if (target == rewind)
startService(new Intent(ACTION_REWIND));
else
if (target == stop)
startService(new
Intent(ACTION_STOP));
else
if (target == eject)
showUrlDialog
();
}
tap(305, 544)Slide16
Existing alternativesRandom TestingCannot perform systematic/exhaustive testingPlatform-specific tools (e.g., hierarchy viewer in Android)Limited to GUI EventsCannot handle third-party GUI widgets
Generating Individual EventsSlide17
Generating Individual EventsOur solutionUse concolic execution to generate data associated with eventsSlide18
FTF
T
tap(int
x, int y){
1 if (x>2 && x<4
){2 if
(y>1 && y<3)3
W1_clicked();4 else5
W2_clicked();6 }else7 W3_clicked();
}
Generating Individual Tap Events
1
7
2
3
5
x>2 && x<4
y>1 && y<3Slide19
Generating Individual Tap Eventstap(1, 5)
F
TF
T
1
7
2
3
5
x>2 && x<4
y>1 && y<3Slide20
Generating Individual Tap Eventstap(1, 5)
Slide21
(x>2 && x<4)Generating Individual Tap Events
tap(1, 5)
Slide22
Generating Individual Tap Eventstap(1, 5)
(x>2 && x<4)(y>1 && y<3)
tap(3, 5)
tap(3, 2)
Slide23
Example: Music Player App
❖
❖
❖
❖
❖
❖
❖
❖
❖
❖
❖Slide24
Generate individual eventsGenerate sequences of events
Two subproblemsSlide25
Generating Sequences of Events
Concatenate individual events generated by concolic
execution.Slide26
Baseline AlgorithmSet of covered branches
S, Set of all event sequences s.t.
each sequence takes a unique path
Baseline algorithm
Goal: cover theseSlide27
Number of sequences generated
for Music Player app by baseline algorithm
Baseline Algorithm Suffers from Path ExplosionSlide28
ACTEve Algorithm
ACTEve
:
A
utomated
C
oncolic
Testing of Eve
nt-driven programsSlide29
ACTEve AlgorithmSet of covered branches
R
s.t. R ⊆ S
S, Set of all event sequences
s.t. each sequence takes a unique path
Baseline algorithm
ACTEve
algorithm
Goal: cover theseACTEve is relatively soundSlide30
Path Subsumption
Maps memory location to values (symbolic or concrete)
Path constraint
Program state in
concolic
executionSlide31
Program
entry
Path
Path
subsumes
Path Subsumption
Note
-
memory map
– path constraint
Slide32
Program
entry
Path
Path
subsumes
Path Subsumption
Note
-
memory map
– path constraint
- Don’t
generate test corresponding to any path that is an extension of
- Only
generate tests corresponding to paths that are extension of
Slide33
Checking path subsumption is very expensive in generalConstraint implication checkMatching memory mapBut, path subsumption can be checked cheaply in special casesRead-only eventsEvents whose mutual ordering does not mattere
tc.Path SubsumptionSlide34
event is does not write to any memory location.
Read-only Events
Program Entry
corresponds to
corresponds to
Path
executed for event sequence
is subsumed by
q
Slide35
❖
❖
❖
❖
❖
❖
❖
❖
❖
❖
❖
Read-only Events
Read-only events are represented as
❖Slide36
ACTEve System ArchitectureSlide37
Empirical StudyApply ACTEve and baseline algorithms
event sequences of length up to
416
concurrently running emulators
time budget of 12
hours
Measured three
metricsrunning timen
umber of feasible pathsnumber
of satisfiability checksSlide38
Empirical ResultsSlide39
Future Work
Widget ExplosionSlide40
Concolic execution to generate individual eventsACTEve: an efficient algorithm for bounded exhaustive testing of event-driven programs Requires only a small fraction (5-36%) of time compared to baseline algorithm
Implementation for Android
Main ContributionsSlide41
Backup slidesSlide42
because
does
not write to any memory location.
Read-only Events
Program Entry
c
orresponds to
event sequence
corresponds to
in
Path
executed for input event sequence
Slide43
Output of Android’s “Hierarchy Viewer” toolA Solution: Use Platform-specific KnowlegeSlide44
A Solution: Use Platform-specific Knowlege
void onTouchEvent(MotionEvent e) { int rawX = (int) e.getX(); int rawY = (int) e.getY(); int x = (rawX – MARGIN) / SIZE; int y = (rawY – MARGIN) / SIZE; if (x >= 0 && x < 3 && y >= 0 & y < 3
) { int cell = x + 3 * y; … }
Output of Android’s “Hierarchy Viewer” toolSlide45
Program Entry
Path
{
is
feasible}
{
is feasible
}
Path
Program Entry
Covered branches
Covered branches
same program location
Path SubsumptionSlide46
Program Entry
Path
{
is
feasible}
{
is feasible
}
Path
Program Entry
Covered branches
Covered branches
same program location
if we explore all paths that extends
, then no need to explore any path that extends
because no additional branch coverage will be obtained.
Path SubsumptionSlide47
Path constraint when PAUSE
button is tapped on
Example: Music Player App