Benjamin Wester Peter Chen and Jason Flinn University of Michigan Speculative Execution Sequential dependent tasks Predict results of Task A to break dependence Execute Task B in parallel Isolate all effects ID: 408519
Download Presentation The PPT/PDF document "Operating System Support for Application..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Operating System Support for Application-Specific Speculation
Benjamin Wester
Peter Chen and Jason
Flinn
University of MichiganSlide2
Speculative Execution
Sequential dependent tasks
Predict results of Task A
to break dependence
Execute Task B in parallelIsolate all effectsCorrect prediction: commitWrong prediction: abort
A
B
B
TIME
2
EuroSys 2011
Predict ASlide3
Speculation Everywhere!
Discrete event simulation
I/O prefetching
Distributed shared memory
Distributed file systemsDeadlock detectionRemote displaysWeb page pre-renderingEuroSys 20113Slide4
Speculation as a Service to Apps
How is this system designed?
In what ways can it be customized for an app?
How can those customizations be specified?
4EuroSys 2011Slide5
Outline
Introduction
Designing Speculation as a Service
ImplementationEvaluation
ConclusionEuroSys 20115Slide6
Design 1: In-App Speculation
Complete semantic info
Predict
arbitrary app operations
Safe operations allowedNo reuse: significant development neededScope is limited: unsafe operations block
EuroSys 20116
App 1
Data
App 2
DataSlide7
OS
Design 2: Generic OS Speculation
Apps need
no modifications
Wide scope: unsafe operations taintLacks semantic understanding of app
Predict system calls onlyHandle application
conservativelyEuroSys 2011
7
App 2
App 1Slide8
Separate Mechanism and Policy
Mechanism
implements isolation
Policy describes customizationsBest of both extremesMechanism built in OSCommon implementationWide scopePolicy specified in ApplicationsExpose semantic informationEuroSys 2011
8Slide9
OS
Design 3: Expose Predictions
Predict
arbitrary app operations
Reuse OS mechanism(with app assistance)Wide
scope for taint propagationLimited semantic info
Speculative external output never allowedCommit on identical
resultsEuroSys 2011
9
App 2
App 1Slide10
OS
Design 4: Expose Safety
Predict arbitrary app operations
Reuse OS mechanism
(with app assistance)Wide scope for taint propagation
More semantic infoAllow safe output
Commit on equivalent results
EuroSys 201110
App 2
App 1Slide11
Customizable Policy
Creation
What tasks are predictable
How to predict themOutput
What output is safe to allowCommitWhich results are acceptable to commitEuroSys 201111Slide12
Outline
Introduction
Designing Speculation as an OS Service
Implementation
EvaluationConclusionEuroSys 201112Slide13
Implementation
Mechanism built in OS
Based on Speculator kernel
Checkpoints & logs processes, files, IPC, etc.
Policies expressed using system call APIEuroSys 201113Slide14
Speculative
Process
Control
Process
s
pec_fork
()
EuroSys 2011
14
TIME
spec_fork
predict A
B
commit
A
abort
B
CSlide15
API Example
int
main() {
int x; int prediction =
get_prediction(); if (
spec_fork() == SPECULATIVE) { x = prediction;
} else { x =
slow_function();
if (equiv
(x, prediction))
commit();
else
abort();
}
set_output_policy
(stdout
, ALLOW);
printf
(“%d”, x);
}
EuroSys 2011
15
Output Policy
Creation Policy
Commit PolicySlide16
Outline
Introduction
Designing Speculation as an OS Service
Implementation
EvaluationConclusionEuroSys 201116Slide17
Evaluation
Can apps effectively use API
to increase parallelism?
Case studies
Predictive application launching in BashSSL certificate checks in FirefoxReplicated service in PBFT-CSEuroSys 201117Slide18
Check command
App 1: Predictive Launching in Bash
EuroSys 2011
18
TIME
Run program
…finished
bwester
$ ▌
…finished
bwester
$
grep foo –r .
▌
grep foo –r .
Creation Policy
:
Use machine learning
to predict user input
Commit Policy
:
Normalize user input before comparison
Output Policy
:
Safe X11 Messages: AllowSlide19
How Much Work Can Be Hidden?
EuroSys 2011
19
TIME
Non-Speculative
45
Speculative
Prompt
Get
cmd
Spec.
Execute
cmd
Execute
Prompt
Execute
cmd
Get
cmdSlide20
How Much Work Can Be Hidden?
EuroSys 2011
20
TIME
Speculative
Non-Speculative
45
U
p to 86% hidden
Prompt
Get
cmd
Spec.
Execute
cmd
Execute
Prompt
Execute
cmd
Get
cmdSlide21
Done
Request page
App 2: Firefox SSL Connections
EuroSys 2011
21
TIME
Web Server
Validation Server
Open https://
Check cert.
Validate
Get session key
GET /
index.html?id
=0123
Creation Policy
:
Predict certificate is valid
Output Policy
:
Alow
SSL connection
Output Policy
:
Block private dataSlide22
Connection Latency Hidden?
EuroSys 2011
22
TIME
Non-Speculative
Speculative
https://
Check cert.
S
ession key
Done
Request
https://
Check cert.
S
ession key
Done
Request
ValidateSlide23
Connection Latency Hidden?
EuroSys 2011
23
TIME
Non-Speculative
Speculative
Avg
60ms hidden
https://
Check cert.
S
ession key
Done
Request
https://
Check cert.
S
ession key
Done
Request
ValidateSlide24
App 3: PBFT-CS Protocol
EuroSys 2011
24
TIME
Send Request
Distributed Replicated
Service
(
Reply, sig1)
Check reply
Check reply
Request
(
Reply, sig2)
Work…
Creation Policy
:
Agree on 1
st
reply
Output Policy
:
PBFT-CS Messages: AllowSlide25
Improved Client Throughput?
EuroSys 2011
25
TIME
Non-Speculative
Speculative
Request1
1
st
Reply
Verify
Request2
1
st
Reply
Request1
1
st
Reply
Verify
Request2
1
st
Reply
Null
requestsSlide26
Improved Client Throughput?
EuroSys 2011
26
TIME
Non-Speculative
Speculative
1.9x Throughput
Request1
1
st
Reply
Verify
Request2
1
st
Reply
Request1
1
st
Reply
Verify
Request2
1
st
Reply
Null
requestsSlide27
Cost of Generic Mechanism
EuroSys 2011
27
App-specific
MechShared OS
MechNon-speculative
Null requestsSlide28
Cost of Generic Mechanism
EuroSys 2011
28
App-specific:
8% faster
App-specific
MechShared OS Mech
Non-speculativeNull requestsSlide29
Conclusion
Mechanism
Common: checkpoints, output buffering, taint propagation
Implemented in OS
PolicyApp-specific: Controls creation, output, and commitImplemented in applicationsDemonstrated with 3 case studiesImproved parallelismSmall overhead relative to app-specific mechanismQuestions?EuroSys 201129