Controlling UserPerceived Delays in ServerBased Mobile Applications MIT Microsoft Research Lenin Ravindranath Jitendra Padhye Ratul Mahajan Hari Balakrishnan Cloud Services Mobile Apps ID: 574037
Download Presentation The PPT/PDF document "Timecard" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Timecard:Controlling User-Perceived Delays in Server-Based Mobile Applications
MIT, Microsoft Research
Lenin Ravindranath, Jitendra Padhye,
Ratul Mahajan, Hari BalakrishnanSlide2
Cloud Services
Mobile AppsSlide3
Response Time Matters
Users are impatientSlide4Slide5
User interaction
Rendering
User-perceived delay
Tightly controlSlide6
App
App
Server
Uplink
Downlink
User interaction
Rendering
Server processing delay
App
processing delay
App
processing delay
Request
t
ransfer delay
Response
t
ransfer delay
User-perceived delay
Tightly controlSlide7
Server
UplinkDownlink
Server delay
C
ontrol
App
App
User interaction
Rendering
Server processing delay
User-perceived delay
Tightly control
Control Server DelaySlide8
Request
ResponseDeadlineServer processing
Controlling the
Server Delay
Trade-off quality of result for processing time
More time to process, better quality results
ServerSlide9
Request
ResponseDeadlineControlling the Server Delay
Worker
Worker
Worker
Worker
Worker
ServerSlide10
Request
ResponseDeadlineControlling the Server Delay
Worker
Worker
Worker
Worker
Worker
Better result
ServerSlide11
Deadline
Controlling the Server DelayServer
Server processing
Servers keep fixed deadlines
No consideration about external delays
Assume constant external delays
Assumed
e
xternal delay
Assumed
e
xternal delaySlide12
Device type
Data sizeTCP stateTCP stateDevice type
Significant Variability in External Delays
App
App
Server
User interaction
Rendering
Server processing
Reading sensors
Radio state
Network
(3G, WiFi, LTE, ..)
RTT, tput
Highly variable
[AppInsight – OSDI ’12]Slide13
Significant Variability in External Delays
ServerClient with high external delaysPoor end-to-end response timeClient with low external delaysDo not produce the best quality result
DeadlineSlide14
Significant Variability in External Delays
ServerServers should adapt to external delaysto control the user-perceived delaySlide15
Significant Variability in External Delays
Server
DeadlineSlide16
Significant Variability in External Delays
ServerAdaptSlide17
Significant Variability in External Delays
ServerUser interactionRendering
End-to-end deadline
Adapt
App
AppSlide18
Timecard
AppApp
ServerSlide19
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Time elapsed since
u
ser interaction
Predicted
downlink
&
app processing
delay
AdaptSlide20
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Desired end-to-end delay
Adapt Processing TimeSlide21
Timecard
AppAppServer
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Adapt Processing Time
Trade-off on
quality of the result
Desired end-to-end delaySlide22
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Adapt Response
Desired end-to-end delaySlide23
Timecard
AppAppServer
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Adapt Response
Desired end-to-end delaySlide24
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Better
quality result
Desired end-to-end delaySlide25
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Time elapsed since
u
ser interaction
Predicted
downlink
&
app processing
delaySlide26
Challenges
ServerApp
AppSlide27
Challenges
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Server
App
UI dispatcher
Web callback
App
Highly asynchronousSlide28
Server Threads
Challenges
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
Transaction
Highly asynchronousSlide29
Challenges
Server
App
App
GetElapsedTime();
No single reference
clock
Variable network delays
a
nd processing delays
PredictRemainingTime(
responseSize
);
Transaction
Highly asynchronousSlide30
Timecard
Server
App
App
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Transaction
Transaction Tracking
TimeSync
Delay PredictorsSlide31
Timecard
App
App Instrumenter
Instrumented
App
Service
Developer
Timecard.dll
GetElapsedTime();
PredictRemainingTime
(
responseSize
);
Desired end-to-end delay
config
App StoreSlide32
Timecard
Server
App
App
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Transaction
Transaction Tracking
TimeSync
Delay PredictorsSlide33
Timecard
Server
App
App
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Transaction
Transaction Tracking
TimeSync
Delay PredictorsSlide34
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TransactionSlide35
Server Threads
Transaction
Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Track across thread boundaries
at the app and at the serverSlide36
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
Track between app and serverSlide37
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TransactionSlide38
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TC
TC
TC
TC
TC
TC
TC
TC
TC
TC
Transaction context (TC)
attached to every thread
Carry along timestamps, transaction informationSlide39
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TC
TC
TC
TC
Event handler
– create new Transaction context (TC)
Timestamp and attach to thread
Asynchronous call
– Pass TC from current thread to callback thread
Pass TC by detouring callbacks [AppInsight – OSDI ‘12]Slide40
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TC
TC
TC
TCSlide41
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TC
TC
TC
TC
TC
HTTP[“x-Timecard-Request”]
TC
Web Request
– pass TC from current thread to server
Encode TC in a special HTTP header
Request handler at server
– Parse TC and attach to server threadSlide42
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TC
TC
TC
TC
TC
TCSlide43
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
Spawn workers
Send response
UI dispatcher
Web callback
Server
App
App
Request
handler
TC
TC
TC
TC
TC
TC
TC
TC
TC
TC
TC
GetElapsedTime();Slide44
Timecard
Server
App
App
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Transaction
Transaction Tracking
TimeSync
Delay PredictorsSlide45
TimeSync
GPS
Sync with Cell Tower
Off my several seconds to minutes
GPS
Does not work indoors
Probing
ProbeSlide46
TimeSync
Instrument app to automatically send probesProbe
Symmetric
Asymmetric
Clock offset
Clock
d
rift
Smartphone clock
Server clockSlide47
TimeSync
ProbeRadio wakeup delay
w
ith large extra delay
Active
Idle
Idle
Network Radio StatesSlide48
TimeSync
Queuing delay
with large extra
delay
Probe
Network traffic
Active
Idle
ActiveSlide49
TimeSync
Network busyRadio idle> 100ms error
m
in RTTSlide50
TimeSync
Optimal time to send the probes Radio is active No network traffic from the phone
Need to be radio-aware and network-aware
ProbeSlide51
TimeSync
ServerApp
AppSlide52
TimeSync
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web request
GPS callback
Event
handler
App
ProbeSlide53
TimeSync
~5ms errorNetwork busyRadio idle
Timecard
m
in RTTSlide54
Timecard
AppApp
Server
GetElapsedTime();
Time elapsed since
u
ser interactionSlide55
Predicting Remaining Time
AppApp
Server
PredictRemainingTime(
responseSize
);
Downlink delay
App processing delaySlide56
Predicting Downlink Delay
Response sizeLatencyThroughputLoss rateTCP window state
Most transfers are short
Median – 3KB
90% percentile – 37KB
Analysis of 4000 apps
Cellular networks
High-bandwidth
,
rare-loss,
high-latency,
99% transfers over HTTP
TCP window state matters -> multiple RTTsSlide57
Predicting Downlink Delay
TCP Window
1 extra RTTSlide58
Predicting Downlink Delay
Response sizeLatencyTCP parametersDetermined by RTTNumber of round tripsSlide59
Predicting Downlink Delay
Use recent estimate of RTTUse time sync probesRead TCP window state
Read TCP window state at server
Middlebox
Do not work with
middleboxes
Split-TCP connection
TCP window state at the middlebox matters
No easy way to estimate
TCP window
state
Build an empirical data-driven model
RTT
Number of round tripsSlide60
Predicting Downlink Delay
PredictRemainingTime(responseSize);LearnDecision tree modelResponse sizeRecent RTTNetwork provider
Bytes already transferred in
the TCP connection
Proxy for TCP window state
at the middlebox
Middlebox
Downlink Delay PredictorSlide61
20 users + controlled experiments1 month, two cities250,000 downloads1 KB to 500KBAT&T, T-Mobile, Sprint, Verizon, Wi-Fi3G, 4G (HSPA+), LTE
Indoors, outdoors, static, walking, drivingPredicting Downlink Delay
Split the data into training and testing
Wi-Fi
Median error -
12 ms
,
90
th
percentile -
31ms
CellularMedian error -
17 ms, 90th percentile - 86 ms Slide62
Predicting App Processing Delay
AppApp
Server
PredictRemainingTime(
responseSize
);Slide63
Predicting Remaining Time
AppApp
Server
PredictRemainingTime(
responseSize
);
Processing Delay Predictor
Downlink Delay PredictorSlide64
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
D
e
r
d
= D – e – r
Desired end-to-end delaySlide65
Mobile ads serviceTwitter analysis serviceDeploymentSlide66
Mobile Ads ServiceContextual ads to mobile apps [Mobisys ’13]Slide67
Mobile Ads Service
Fetch and process ads for keywords
Extract keywords
Render ad
Send keywords
Best AdSlide68
Mobile Ads Service
Extract keywords
Render ad
Send keywords
Ad
Ad provider
Ad provider
Ad provider
Ad provider
Ad provider
A
K
eywordsSlide69
Mobile Ads Service
Extract keywords
Render ad
Send keywords
AdSlide70
Mobile Ads Service
Extract keywords
Render ad
Send keywords
Ad
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Adapt Processing TimeSlide71
Mobile Ads Service
~200,000 transactionsSignificant variability in external delay
Within 50ms of the target delay 90% of the time
With Timecard
Without Timecard
Target
delaySlide72
Mobile ads serviceTwitter analysis serviceDeploymentSlide73
Twitter Analysis Service
Fetch from Twitter and analyze
Get keyword
Parse and render
Send keyword
Score and tweetsSlide74
Twitter Analysis Service
Fetch from Twitter and analyze
Get keyword
Parse and render
Send keyword
Score and tweetsSlide75
Twitter Analysis Service
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Adapt Processing Time
Adapt Response
Get keyword
Parse and render
Send keyword
Score and tweets
10KB to 50 KBSlide76
Twitter Analysis
Service~150,000 transactionsAdapt server processing time in steps of 150msWith Timecard
Without Timecard
Target
delay
Tightly control end-to-end delay with TimecardSlide77
0.1% runtime overheadLess than 1% memory overheadLess than
1% network overheadNegligible battery overheadTimecard OverheadSlide78
LimitationsBootstrappingTraining the downlink and app processing delay predictorsLearn app processing delay for every transaction type
Turn off adaptation till enough data is collectedComplex transactionsMultiple parallel or serial requestsElapsed time will work!Need to model complex interactions or need developer helpSlide79
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Adapt Processing Time
Adapt Response
End-to-end deadlineSlide80
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Adapt Resources Used
End-to-end deadlineSlide81
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Request Prioritization
End-to-end deadlineSlide82
Timecard
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Deadlines
SchedulerSlide83
Controlling end-to-end delaysElapsed timePrediction on remaining timeCan be applied in several context
Take-AwayAdapt quality of the paperSOSP Deadline
GetElapsedTime();
PredictRemainingTime(
paper
);Slide84
Timecard
AppApp
Server
GetElapsedTime();
PredictRemainingTime(
responseSize
);
Time elapsed since
u
ser interaction
Predicted
downlink
&
app processing
delaySlide85
BackupSlide86
Predicting App Processing Delay
AppResponse sizeCorrelated with response sizeAnalysis of AppInsight data1653 types of transactions
Response size (KB)
Processing delay (ms)Slide87
Predicting App Processing Delay
AppResponse size
Parsing delay
Correlated with data size
Rendering delay
Correlated with data size
Dependent on device type
Processing speed
Correlated with response size
Analysis of AppInsight data
1653 types of transactionsSlide88
Predicting App Processing Delay
AppPredictRemainingTime(responseSize);
Processing Delay Predictor
Learn
Decision tree model
Response
size
Device type
Analysis of AppInsight data
1653 types of transactions
Median error -
8ms
90
th
percentile error -
100ms