Collaborative - PowerPoint Presentation

Slide1

Collaborativeruntime verificationwith tracematches

Eric BoddenLaurie HendrenPatrick LamOndrej LhotakNomair A. Naeem

McGill University

University of WaterlooSlide2

ProblemIdeally, runtime verification code should be included in deployed programs:

Allows for easier debuggingActual usage vs. test case coverageCurrent runtime monitoring approaches do not scale well enough.2Here:

TracematchesSlide3

A common programming problem

Collection c =

Collections.synchronizedCollection

(

myC

);

synchronized

(c) {

}

Iterator

i = c.iterator(); while (i.hasNext()) foo(i.next());

3Slide4

Tracematch "ASyncIteration"

tracematch(Object c) { sym sync

after returning(c):

call

(* Collections.synchr*(..));

sym

asyncIter

before

:

call(* Collection+.iterator()) && target(c) &&

if(!Thread.holdsLock(c)); sync asyncIter { System.err.println( "Iterations over "+c+" must be synchronized!" ); }}4Slide5

5

Static Optimizations (ECOOP 2007)Slide6

Static Optimizations (ECOOP 2007)

Quick check: Eliminate incomplete tracematchesPointer analysis:Retain “consistent sets of instrumentation points”Brings overhead under 10% in most cases.However, some overheads still exceed 150%!Goal: 10% overhead in all cases6Slide7

7

C

ollaborative

runtime verification

Spatial

partitioningSlide8

8

Spatial partitioning in detailFirst of all, identify multiple probes:A set of instrumentation points (shadows) that could potentially lead to a matchFind such sets of shadows using flow-insensitive points-to analysisSlide9

Identifying probes

o1

o2

asyncIter

(c=c2)

asyncIter

(c=c3)

sync(c=c1)

9

ProbeSlide10

10

CompletenessSlide11

11

Temporal

partitioning









Problem: Hot shadowsSlide12

Could switching probes on and off lead to false positives?

12No, we can safely enable a probe anytime due to tracematch semantics.Opposed to e.g. LTL always match against a suffix of the execution trace.Can also disable anytime.Just have to make sure we discard bindings.

sync

aSyncIter

skip(

aSyncIter

)

*Slide13

Code generation for probe switching

asyncIter

(c=c3)

sync(c=c1)

13

asyncIter

(c=c2)

sync(c=c5)

asyncIter

(c=c4)

sync(c=c1)

0

1

2

3

0

4

0

1

2

0

1

2

3

4Slide14

Benchmarks

BenchmarkTracematchprobesInitial additional runtimeantlrReader420.6%chart

FailSafeIter74220.6%

lucene

HasNextElem

6

11.9%

pmd

FailSafeIter

42679.0%pmdHasNext32158.1%14Ran each benchmark/

tracematch combination with one probe enabled at a timeMeasured relative runtime overheadECOOP ’07 benchmarks with largest overheadsSlide15

Overheads after spacial partitioning

15Slide16

Future workImplement temporal partitioning

Requires probabilistic foundationTry this out on a larger scaleNeed Java programs with a large user base, willing to cooperateTry using JVM support to find hot probesProduction JVMs already compute statisticsWould enable more efficient probe switchingEliminate super-hot shadows through better static analysis16Slide17

Conclusion

Sound collaborative RV is possible using tracematchesCan construct probes using a flow-insensitive points-to analysisApproach works for some programs but very hot shadows can still be bottlenecksFound a heuristic to statically identify shadows with potentially high runtime impactFurther static optimizations probably more promising17Slide18

Thank youThank you for listening and the entire

AspectBench Compiler group for their enduring support!Download our tool, examples and benchmarks at:www.aspectbench.org18Slide19

Past optimizationsSpecialized code generation for runtime monitors

Leak eliminationIndexingBrings overheads down from “infeasible” to “same order of magnitude” in most cases19Slide20

pmd/HasNext

pmd/FailSafeIterhitsshadow

hitsshadow

1592253

tracematch$2$hasNext@31538

1591670

tracematch$2$call_next@31539

1591670

tracematch$2$next@31539

204760

tracematch$2$call_next@32625245224tracematch$2$hasNext@32624

10498tracematch$2$create_iter@32135204760tracematch$2$next@326257831tracematch$2$update_source@3197116764tracematch$2$hasNext@321367820tracematch$2$call_next@32137ProblemThere are some very few shadows which are extremely hot.20Why is pmd/HasNext

so much slower?Slide21

Debug outputOur system can generate debug output…

21OutputMeaningDbinding is created

dbinding is destroyedSlide22

Pmd/FailSafeIterOutput over the entire run:

DDIn particular: Calls to next() cause no new bindings to be created.22Slide23

Pmd/HasNextOutput over the entire run:

…DdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDddDddDddDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDddDddDdDdDdDdDdDdDdDddDddDddDddDddDddDddDdDdDdDdDdDdDdDdDdDdDdDdDdDdDddDddDddDdDdDdDdDdDdDdDddDddDdDdDdDdDdDdDdDdDdDdDdDdDdDdDdDddDddDdDdDdDd…1796765 bindings created, 1796739 bindings destroyed23Slide24

24

Identifying potentially expensive shadows

next

next

skip(next)

skip(

hasNext

)

create

update

next

skip(create)skip(update)

next

update

skip(next)

skip(create)Slide25

First it was looking good…

25(Initial overhead was 20.6%)Slide26

Prototype implementation

26Compiler determines the set of all probes. Each probes is assigned a unique numberGenerate array mapping probes to shadowsGenerate a Boolean array over all shadowsGuard each shadow by its Boolean flagRuntime API allows for reconfigurationEnable/disable probes (switches flag)Currently no network support yetSlide27

Probes &

RelationshipsProbes &Relationships

Probe 4Collaborative runtime verification

27

Probe info

Statistics

Relationships

Probe 1

Client 3

Probe 1

Client 1

Probe 5Client 2

ServerSlide28

28