Are We There Yet in Industrial Cases Wing Lam Zhengkai Wu Dengfeng Li Wenyu Wang Tao Xie University of Illinois at UrbanaChampaign USA Haibing Zheng Hui Luo ID: 634601
Download Presentation The PPT/PDF document "Record and Replay for Android:" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Record and Replay for Android: Are We There Yet in Industrial Cases?
Wing Lam Zhengkai Wu Dengfeng Li Wenyu Wang Tao XieUniversity of Illinois at Urbana-Champaign, USAHaibing Zheng Hui Luo Peng Yan Yuetang Deng Tencent, Inc., China
FSE
INDUSTRY 2017Slide2
Estimated 9.2 million Android app developers1
Tools like Monkey2 exist to help developers automatically test their appRecord and replay tools help developers test specific use cases:Regression testing on two different versions of the appCompatibility testing on a version of the app but on different devicesPerformance testing on a version of the app many timesAndroid testing tools21: http://www.businessofapps.com/12-million-mobile-developers-worldwide-nearly-half-develop-android-first/2: https://
developer.android.com/studio/test/monkey.html
Are the existing record and replay tools good enough in practice
?Slide3
Our workQualitative study of
popular record-and-replay tools Quantitative study of selected record-and-replay tools based on three main aspects: Ability to reproduce common usage scenarios on the same deviceon different devicesSpace overhead3Slide4
Selected appsSelection based on their popularity
and category 4Slide5
Desirable characteristicsFive highly desirable characteristics:
coordinate sensitive widget sensitive state sensitivetiming sensitive custom OS5Four secondary characteristics:access to the source code of the app under testinstrumenting the approoting the deviceopen sourcedSlide6
Coordinate sensitiveEvents are recorded based on the coordinates
of the actions on the GUI screenExample trace: TOUCH|{'x':513,'y':1692,'type':'downAndUp',}6Image from http://tekeye.biz/wp-content/uploads/2013/03/android_coordinate_system.pngSlide7
Widget sensitiveEvents are recorded based on the widgets (e.g., buttons, text fields) that the
actions interacted withExample trace:TOUCH|{'id':’R.id.loginButton','type':'downAndUp'}7Image from https://www.tutorialspoint.com/android/images/ui_control.jpgSlide8
State sensitiveTool can be replayed successfully only if the device’s state is same as when it was recorded
Typical state-insensitive tools reset the app before record and replay8Slide9
Timing sensitiveTool automatically sets the timing between recorded
eventsExample trace:WAIT|{'seconds':2.0,}9Gif from https://www.youtube.com/watch?v=X2otzqZvZ4MSlide10
Characteristics of each tool
****** - Desirable characteristics10Slide11
Inapplicable tools
11There exist many record-and-replay tools, but few are available for developers to actually use in practiceUnfortunately, none of the tools that we find exhibit all of the desirable characteristicsSlide12
Our work
Qualitative study of popular record-and-replay tools Quantitative study of selected record-and-replay tools based on three main aspects: Ability to reproduce common usage scenarios on the same deviceon different devicesSpace overhead12Slide13
Selected tools and apps
Tools: Apps:* - Desirable characteristics**
*
*
*
13Slide14
Reproducing common usage scenariosRERAN is the
only tool able to record and replay all scenarios“-” : the tool produces unstable results14Slide15
Replaying robustnessRecorded traces from common usage scenarios replayed
on different-resolution devicesAll three tools performed differently; Appetizer did best but still fails because it cannot support dragging eventsNone of these tools are able to replay complicated scenarios on different screen sizes successfully15Slide16
Space overhead (trace size in kB)Traces
produced by RERAN occupy about 67 times the space that traces from monkeyrunner occupy* Size is in kB* Size is in kB16Effectiveness of RERAN comes at the cost of space overheadSlide17
ConclusionQualitative study of popular record-and-replay tools for Android apps
Quantitative study of selected record-and-replay tools for three popular industrial Android appsStudy results highlighting strengths and weaknesses of tools and identifying future tool-research directions17Slide18
Thank you! Questions?18Slide19
ConclusionQualitative study of popular record-and-replay tools for Android apps
Quantitative study of selected record-and-replay tools for three popular industrial Android appsStudy results highlighting strengths and weaknesses of tools and identifying future tool-research directions19