Thakur Rathijit Sen Ben Liblit Shan Lu University of WisconsinMadison Workshop on Dynamic Analysis 2009 Cooperative Crug Isolation Cooperative ID: 474443
Download Presentation The PPT/PDF document "Aditya" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Aditya Thakur Rathijit Sen Ben Liblit Shan LuUniversity of Wisconsin–Madison
Workshop on Dynamic Analysis 2009
Cooperative
Crug
IsolationSlide2
Cooperative Crug Isolation
read(
x
)
write(
x
)
Thread 1
Thread 2
Race
!
read(
x)
read(x)
write(x)
Thread 1
write(
x)
Thread 2
Atomicity violation!
(concurrency bug)Slide3
Cooperative Crug Isolation
threaded.exe
file.in
threaded.exe
file.in
L
J
developer
user
Non-determinism!
More cores
More threads
L
L
L
L
L
L
L
More
crugsSlide4
Cooperative Crug IsolationSlide5
LCooperative Crug Isolation
unlock(
mut
);
lock(
mut
);
Thread 1
mut
= NULL;
Thread 2
Global variables are shown in
bold.
Simplified crug from PBZIP2JSlide6
Cooperative Crug IsolationGlobal variables are shown in bold.
Identify root cause of
crug
unlock(
mut
);
lock(
mut
);
Thread 1
mut
= NULL;
Thread 2Slide7
Cooperative Crug Isolation
Not scalable,High overhead
Report benign
crugs
Target specific type of
crugs
and synchronization
Current techniquesSlide8
Cooperative Crug Isolation
Scalable,
Low overhead
Does not report
benign
crugs
Multiple types of
crugs
and synchronizationSlide9
ShippingApplicationCooperative
Crug Isolation
Bug
Isolation
ƒ
ƒ
€
‚
Program
Source
Compiler
Sampler
Predicates
Counts
&
J
/
L
Statistical
Debugging
Top
bugs
with
likely causesSlide10
Cooperative Crug Isolation
Bug
Isolation
unlock(
mut
);
lock(
mut
);
Thread 1
mut
= NULL;
Thread 2
unlock(mut);
lock(mut);
Thread 1
mut
= NULL;Thread 2
CBI
predicates
inadequate for
crug
isolation.
Values of predicates same for successful and failing runs.Slide11
Cooperative Crug Isolation
Bug
Isolation
unlock(
mut
);
lock(
mut
);
Thread 1
mut
= NULL;
Thread 2
CBI sampling inadequate for crug isolation.
Sampling thread-local, independent.Slide12
Cooperative Crug Isolation
Bug
Isolation
CBI was unable to diagnose
crugs
in any of the benchmarks used.
No bug predictors reported!Slide13
Cooperative Crug IsolationCCI extends the CBI framework to target crugs
New predicate capturing interleaving events
New cross-thread sampling schemeSlide14
Cooperative Crug IsolationPredicate Design
unlock(
mut
);
S:
lock(
mut
);
Thread 1
mut
= NULL;
Thread 2
remote
S
is true
L
local
S is trueJSlide15
Predicate InstrumentationAt runtime,maintain hashtable which mapsaddresses to thread id which last accessed it
Address
Thread Id
0xb1ab1a
1
0xf00f00
2
0xb1af001Slide16
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);Slide17
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);curTid is thread id of currently executing threadSlide18
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);Check if curTid was the thread which previously accessed xSlide19
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);Set differs to true if it was notSlide20
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);Update the hashtableSlide21
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);Increment counter for predicateat SSlide22
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);Execute block atomicallySlide23
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid
);
lock(
glock);
unlock(glock);Handles accesses through pointers. No need for static pointer analysis.Slide24
Predicate Instrumentationaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
,
curTid);
lock(
glock);
unlock(glock);curTid is thread id of currently executing thread
Check if curTid was the thread which previously accessed xSet differs to true if it was not
Increment counter for predicateat S
Execute block atomically
Update the hashtableSlide25
Sampling Mechanismaccess(
x);
record(S
,
differs);
differs =
test_and_insert
(&
x
, curTid);
lock(
glock
);unlock(
glock);If(
gsample == 0)access(
x);
gsample
= curTid; insert(&x, curTid
);
else
if(
gsample
==
curTid
)
gsample
= 0; clear();
Is sampling on?
Turn on sampling
Update
hashtable
Stop sampling, clear
hashtable
Did current thread initiate sampling
Sampling not on
Sampling already onSlide26
Sampling Mechanismlock(
mut);
Thread 1
Address
Thread Id
Hashtable
gsample
= 0Slide27
Sampling Mechanismlock(
mut);
Thread 1
Thread 2
Address
Thread Id
&
mut
1
Hashtable
gsample
= 1Slide28
Sampling Mechanismlock(
mut);
Thread 1
mut
= NULL;
Thread 2
Address
Thread Id
&
mut
2
Hashtable
gsample
= 1Slide29
Sampling Mechanismunlock(
mut);
lock(
mut
);
Thread 1
mut
= NULL;
Thread 2
S:
Address
Thread Id
&x2
Hashtable
gsample = 1Record remote
S is trueSlide30
Sampling Mechanismunlock(
mut);
lock(
mut
);
Thread 1
mut
= NULL;
Thread 2
S:
Address
Thread Id
Hashtable
gsample
= 0Stop samplingSlide31
Experimental EvaluationBenchmarks usedApache HTTP server, PBZIP2SPLASH-2: FFT, LU
Machine used
dual-core Intel P4
Questions answered
Runtime overhead
Accuracy of predictorsSlide32
Runtime OverheadBenchmarkNo sampling
Sampling
Apache
25%
2%
PBZIP2
200%
7%FFT650%25%LU
1,300%800%
Overhead compared to
uninstrumented code
Low overheads for both real-world applications
Large difference between no sampling and sampling.Slide33
Predictor AccuracyPredictorFunction
R: buf
->
outcnt
+=
len
ap_buffered_log_writer
()
Apache
PredictorFunction
R
:
pthread_mutex_unlock(fifo->
mut);consumer_decompress()
PBZIP2remote predicateSlide34
Predictor AccuracyPredictorFunction
R: Global->
finishtime
=finish
SlaveStart
()
R
: Global->
initdonetime=
initdoneSlaveStart
()
R
: printf(“..”,Global->transtime[0]…)main()
L: malloc
(2*(rootN-1)*sizeof(double));
SlaveStart()
FFT
PredictorFunction
R: Global->
rf=rf
OneSolve
()
L
:
(Global->start).
gsense
=-
lsense
;
OneSolve
()
LU
local predicateSlide35
ConclusionCCI is a low-overhead, scalable approach for
root cause analysis of crugs
Effective on two widely-deployed applications
Simple predicates are effective
because
of the use of statistical modelsSlide36
Next time on What other events are useful for crug isolation?Scope for static analysis to help?
Other cross-thread sampling mechanisms
(e.g.
bursty
sampling)?
Crug
isolation to crug
tolerance?
Thank you!