Automated AtomicityViolation Fixing Guoliang Jin Linhai Song Wei Zhang Shan Lu and Ben Liblit University of WisconsinMadison AFix 1 Multicore era is coming already here Programmers struggle ID: 764484
Download Presentation The PPT/PDF document "Automated Atomicity-Violation" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
AutomatedAtomicity-ViolationFixing Guoliang Jin, Linhai Song, Wei Zhang, Shan Lu, and Ben LiblitUniversity of Wisconsin–Madison AFix 1
Multicore era is coming already hereProgrammers struggle to reason about concurrencyMore and more concurrency bugsMany concurrency bugs can be automatically detectedBut bugs need to be fixed Needs to Find Concurrency Bugs Needs to Find and Fix Concurrency Bugs 2 Thread 1 if ( ptr != NULL) { ptr->field = 1;} Thread 2 ptr = NULL; Segmentation Fault
Understand bug1……Understand bugn Bug-fixing process is lengthy and resource consumingNearly 70% of patches are buggy in their first releases Automated fixing is desired, but difficult in general Bug-fixing 3 Understand a bug Generate a patch Review& testthe patchCorrectness Performance Readability
Concurrency bugs are feasible to be fixed automaticallyProgram is correct in most interleavings. Only need to remove some bad interleavings. Automated Concurrency-Bug Fixing 4 Thread 1 if ( ptr != NULL) { ptr->field = 1;}Thread 2ptr = NULL; Thread 1 if ( ptr != NULL) { ptr->field = 1;} Thread 2 ptr = NULL; Thread 1 if ( ptr != NULL) { ptr ->field = 1; } Thread 2 ptr = NULL ; Segmentation Fault
Why atomicity-violation bugs?One of the most common types of concurrency bugStrategyStatically adding locks to remove buggy interleavings.Goal Automate the whole bug-fixing processProvides best-effort atomicity-violation patchesCorrectnessPerformanceReadability AFix : Automated Atomicity-Violation Fixing 5
Input from CTrigger AFix Overview 6 Bug understanding Manual Bug Fixing Progress
A single-variable atomicity-violation detection & testing tool It reports a list of buggy instruction triplesAbbreviated as {(p1, c1, r1 ), …, (pn, cn, rn)} CTrigger Bug-Detector Review 7 Thread 1 if (ptr != NULL) { ptr ->field = 1;} Thread 2ptr = NULL; p revious access c urrent access r emote access
Input from CTrigger AFix Overview patch n patch1(p1, c1, r1)…...…(p n, cn, rn) merged patch 1 adding runtime support patchtesting ……… mergedpatch m … 8 Patch generation Bug understanding Patch testing Manual Bug Fixing Progress
MotivationOverviewAFix One bug patchingPatch MergingRuntime supportPatch testingEvaluationConclusion Outline Motivation Overview AFix One bug patching Patch MergingRuntime supportPatch testingEvaluationConclusion Motivation Overview AFix One bug patchingPatch Merging Runtime supportPatch testingEvaluationConclusion 9
Make the p-c code region mutually exclusive with rPut p and c into a critical sectionPut r into a critical sectionSelect or introduce a lock for the two critical sections One Bug Patching ( p , c, r) Patching 10 p c r
A naïve solutionAdd lock on edges reaching pAdd unlock on edges leaving cPotential new bugsCould lock without unlockCould unlock without locketc. Put p and c into a Critical Section: naïve p c p c 11 p c p c
Assume p and c are in the same function fStep 1: find protected nodes in critical section In f’s CFG, find nodes on any p c pathStep 2: add lock operationsunprotected node protected nodeprotected node unprotected nodeAvoid those potential bugs mentioned Put p and c into a Critical Section: AFix p c 12
p and c adjustment when they are in different functionsObservation: people put lock and unlock in one functionFind the longest common prefix of p’s and c ’s stack tracesAdjust p and c accordingly p and c Adjustment void newlog(){ … close(); open(); … }void close () { … log = CLOSE; } void open () { … log = OPEN ; } void newlog () { … p: close (); c: open (); … } p: c : … newlog () close() … newlog () open() 13
( p , c, r) Patching: put r into a critical section Lock-acquisition before r, lock-release after r Only if r cannot be reached from the p–c critical sectionfpc() {lock(L1)p...r…c unlock(L1) }case 1 f pc() {lock(L1)p...foo() {…r}…cunlock(L1)} case 2r’s call stack: … fpc foo …r 14
( p , c, r) Patching: select or introduce a lock Use the same lock for the critical sections Lock type: Lock with timeout : in case of potential new deadlockReentrant lock : in case of recursion Otherwise: normal lockLock instance:Global lock instances are easy to reuse 15
c1 r1 p1 p2 c 2, r2 void buf_write() { int tmp = buf_len + str_len; if (tmp > MAX) return; memcpy (buf[buf_len], str , str_len ); buf_len = tmp ;} Patch Merg ing 16 p1 c1 p2 r1 c2, r2 Too many lock/unlock operations Potential new deadlocks May hurt performance and readability One programming mistake can lead to multiple bug reports They should be fixed all together
Redundant patch, when p1 –c1, p2–c2 critical sectionsare in the same function: redundant when one protected region is a subset of the otherare in different functions: consulting the stack trace again Patch Merging: redundant patchlock(L1)p1lock(L2)p2c2unlock(L2)c1unlock(L1)lock (L1)r1unlock(L1)lock(L2)r2unlock(L2)lock(L1)p1p2c2c1unlock(L1)lock(L1)r2unlock(L1 ) 17
Related patch Merge if p, c, or r is in some other patch’s critical sections Patch Merging: related patch lock (L1)p1lock(L2)p2c1unlock(L1)c2unlock(L2)lock(L1)r1 unlock(L1) lock(L2) r2unlock(L2) lock(L1)p1p2c1c2unlock(L1)lock(L1)r2unlock(L1) 18
Runtime support to handle deadlockLightweight patch-oriented deadlock detectionWhether timeout is caused by potential deadlock? Only detect deadlocks caused by the patchesHas low-overhead, and suitable for production runsHelp patch refinementTraditional deadlock detectionIn-house patch testing Runtime Support and Testing 19
Motivation Overview AFixOne bug patchingPatch MergingRuntime supportPatch testingEvaluationConclusion Motivation Overview AFix One bug patchingPatch MergingRuntime supportPatch testing EvaluationConclusion Outline 20
Evaluation: Overall Patch Quality Bug NaïveUnmergedMergedManualApache-- MySQL 1 -MySQL 2- Mozilla 1-Mozilla 2- Cherokee- FFT- - PBZIP2- - - Patched failure rates: 0% (except PBZIP2 and FFT) Patched overheads: <0.6% (except PBZIP2 ) With timeout triggered deadlock detection 21
Conclusion Atomicity v iolations are feasible to be fixed automatically By removing bad interleavings Must be careful in the detailsUse some heuristics, and excellent results in practiceCompletely eliminates detected bugs in targeted classOverheads too low to reliably measureProduces small, simple, understandable patchesFuture research should do detector and fixer co-design 22
Questions about AFix*? *disclaimer from Afix: “I represent humans’ efforts towards fixing the world automatically using tools. However, the world is so imperfect that I do not know whether the world is Fully fixable, thus I make no 100% guarantee.” 23