A Static Binary Instrumentation Threading Model for Fast Memory Trace Collection PowerPoint Presentation, PPT - DocSlides

A Static Binary Instrumentation Threading Model for Fast Memory Trace Collection

Michael Laurenzano

1

, Joshua Peraza

1

, Laura Carrington

1

,

Ananta

Tiwari

1

, William A. Ward

2

, Roy Campbell

2

1

Performance Modeling and Characterization (

PMaC

) Laboratory, San Diego Supercomputer Center

2

High Performance Computing Modernization Program (HPCMP), United States Department of Defense

Memory-driven HPC

Many HPC applications are memory bound

Understanding application requires understanding memory behavior

Measurement? (e.g. timers or hardware counters)

Measuring at fine grain with reasonable overheads & transparently is

HARD

How to get sufficient detail (e.g., reuse distance?)

Binary instrumentation

Obtains low-level details of address stream

Details are attached to specific structures within the application

Convolution Methods

map

Application Signatures

to Machine Profiles produce performance prediction

HPC Target

System

Characteristics of HPC system – Machine Profile

HPC Target

System

Machine Profile

–

characterizations of the rates at which a machine can carry out fundamental operations

Measured or projected via simple benchmarks on 1-2 nodes

of the system

HPC

Application

Requirements of

HPC Application – Application Signature

PMaC

Performance/Energy Models

Performance of Application on Target system

HPC

Application

Application signature

–

detailed summaries of the fundamental operations to be carried out by the application

Collected via trace

tools

Performance Model

– a calculable expression of the runtime, efficiency, memory use, etc. of an HPC program on some machine

Runtime Overhead is a Big Deal

Real HPC applications

Relatively long runtimes: minutes, hours, days?

Lots of CPUS: O(10

7

) in largest supercomputers

High slowdowns create problems

Too long for queue

Unsympathetic administrators/managers

InconvenienceUnnecessarily use resourcesPEBIL = PMaC’s Efficient Binary Instrumentation for x86/Linux

What’s New in PEBIL?

It can instrument multithreaded code

Developers use

OpenMP

and

pthreads

!

x86_64 only

Provide access to thread-local instrumentation data at runtime

Supports turning instrumentation on/offVery lightweight operationSwap nops with inserted instrumentation code at runtimeOverhead close to zero when all instrumentation is removed

Binary Instrumentation in HPC

Tuning and Analysis Utilities (TAU) –

Dyninst

and PEBIL

HPCToolkit

–

Dyninst

Open

SpeedShop – DyninstIntel Parallel Studio – PinMemcheck memory bug detector – Valgrindvalgrind

–-leak-check=yes ...Understanding performance and energyMany research projects (not just HPC)BI used in 3000+ papers in the last 15 years

Binary Instrumentation Basics

Memory Address Tracing

Original

Instrumented

0000c000 <

foo

>:

c000: 48 89 7d f8

mov

%rdi,-0x8(%

rbp) c004: 5e pop %

rsi

c005: 75 f8 jne

0xc004 c007: c9 leaveq c008: c3 retq

0000c000 <foo>: c000: // compute -0x8(%rbp) and copy it to a buffer c008: 48 89 7d f8

mov %rdi,-0x8(%rbp) c00c: // compute (%rsp) and copy it to a buffer c014: 5e pop %rsi c015: 75 f8

jne 0xc00c c017: c9 leaveq c018: c3 retq

Enter Multithreaded Apps

All threads use a single buffer?

Don’t need to know which thread is executing

A buffer for each thread?

Faster. No concurrency operations needed

More interesting. Per-thread behavior != average thread behavior

PEBIL uses the latter

Fast method for computing location of thread-local data

Cache that location in a register if possible

0000c000 <

foo>: c000: // compute -0x8(%rbp) and copy it to a buffer c008: 48 89 7d f8

mov %rdi,-0x8(%

rbp)

c00c: // compute (%rsp) and copy it to a buffer c014: 5e pop %rsi c015: 75 f8 jne 0xc00c

c017: c9 leaveq c018: c3 retq

Thread-local Instrumentation Data in PEBIL

Provide a large table to each process (2M)

Each entry is a small pool of memory (32 bytes)

Must be VERY fast

Get thread id (1 instruction)

Simple hash of thread id (2 instructions)

Index table with hashed id (1 instruction)

Assume no collisions (so far so good)

Hash Function

t

hread

1 id

thread 2 id

thread 3 id

thread 4 id

Thread-local memory pools

thread 4’s memory pool

Caching Thread-local Data

Cache the address of thread-local data

Dead registers are known at instrumentation time

Is there 1 register in a function which is dead everywhere?

Compute thread-local data address only at function [re]entry

Should use smaller scopes! (loops, blocks)

Significant reductions

Other x86/Linux Binary Instrumentation

Tool Name

Static

or

Dynamic

Thread-local

Data Access

Threading

Overhead

Runtime overheadPin1DynamicRegister stolen from program, program JIT-compiled around that lost registerVery lowMedium

Dyninst2EitherCompute thread ID (layered function call) at every pointHighVaries

PEBIL3

Static

Table + fast hash function (4 instructions), cache result in dead registersLowLow1Pin: Building Customized Program Analysis Tools with Dynamic Instrumentation.

Luk, C., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Vijay

Janapa Reddi, and Hazelwood, K. ACM SIGPLAN Conference on Programming Language Design and Implementation, 2005.2An API for Runtime Code Patching. Buck, B. and Hollingsworth, J. International Journal of High Performance Computing Applications, 2000.3

PEBIL: Efficient Static Binary Instrumentation for Linux. Laurenzano, M., Tikir, M., Carrington, L. and Snavely, A. International Symposium on the Performance Analysis of Systems and Software, 2010.

Runtime Overhead Experiments

Basic block counting

Classic test in binary instrumentation literature

Increment a counter each time a basic block is executed

Per-block, per-process, per-thread counters

Memory address tracing

Fill a process/thread-local buffer with memory addresses, then discard those addresses

Interval-based sampling

Take the first 10% of each billion memory accesses

Toggle instrumentation on/off when moving between sampling/non-sampling

Methodology

2 quad-core Xeon X3450, 2.67GHz

32K L1 and 256K L2 cache per core, 8M L3 per processor

NAS Parallel Benchmarks

2 sets:

OpenMP

and MPI,

gcc

/GOMP and

gcc/mpich8 threads/processes: CG, DC (omp only), EP, FT, IS, LU, MG4 threads/processes: BT, SPDyninst 7.0 (dynamic)

Timing started when instrumented app begins runningPin 2.12PEBIL 2.0

Basic Block Counting (MPI)

All results are average of 3 runs

Slowdown relative to un-instrumented run

1 == no slowdown

Basic Block Counting (OpenMP

)

Y-axis = log-scale slowdown factor

Dyninst

thread ID lookup at every basic block

Threading Support Overhead

(BB Counting)

Memory Tracing (MPI)

Slowdown relative to un-instrumented application

Tool

BT

CG

EP

FT

IS

LU

MG

SPMEANPEBIL14.934.182.173.532.85

6.084.77

4.13

5.33Pin5.894.432.913.893.544.674.852.48

4.08Dyninst22.4413.765.649.537.2512.9015.31

10.1912.12

Memory Tracing (OpenMP

)

Instrumentation code inserted at

every

memory instruction

Dyninst

computes thread ID at every

memop

Pin runtime-optimizes instrumented code

Lots of opportunity to optimize

ToolBTCGDCEPFTISLUMG

SP

MEAN

PEBIL16.646.052.002.556.215.9410.5510.18

9.327.71Pin6.194.423.013.013.855.56

5.895.264.774.66Dyninst???862.86530.55448.89921.89

752.181759.241555.76???975.90**

30s

 7h45m

Interval-based Sampling

Extract useful information from a subset of the memory address stream

Simple approach: the first 10% of every billion addresses

In practice we use a window 100x as small

Obvious: avoid processing addresses (e.g., just collect and throw away)

Not so obvious: avoid

collecting

addresses

Instrumentation tools can disable/re-enable instrumentation

PEBIL: binary on/off. Very lightweight, but limitedPin and Dyninst: arbitrary removal/reinstrumentation. Heavyweight, but versatileSampling only requires on/off functionality

Sampled Memory Tracing (MPI)

PEBIL always improves, and significantly

Pin usually, but not always improves

Amount and complexity of code re-instrumented during each interval probably drives this

Dyninst

never improves

P

Tool

BT

CG

EP

FT

IS

LUMGSPMEANPEBIL Full14.934.18

2.173.532.856.084.774.135.33PEBIL 10%4.36

2.081.481.771.702.972.201.732.28Pin Full5.89

4.432.913.893.544.674.852.484.08

Pin 10%5.425.012.612.743.195.054.512.98

3.93Pin Best

5.424.432.61

2.743.19

4.674.512.48

3.75Dyninst Full = Best

22.4413.76

5.649.537.25

12.9015.31

10.1912.12

Sampled Memory Tracing (OpenMP

)

Tool

BT

CG

DC

EP

FT

IS

LU

MGSPMEANPEBIL16.646.052.00

2.556.21

5.94

10.5510.189.327.71PEBIL 10%4.592.751.781.59

2.432.613.293.363.292.85Pin Full6.194.42

3.013.013.855.565.895.264.774.66Pin 10%

6.044.483.592.803.053.8810.266.478.96

5.50Pin Best6.044.423.012.803.053.885.89

5.264.77

4.35Dyninst Full = Best*

???862.86

530.55448.89921.89

752.181759.24

1555.76???975.90**

Conclusions

New PEBIL features

instrument multithreaded binaries

Turn instrumentation on/off

Fast access to per-thread memory pool to support per-thread data collection

Reasonable overheads

Cache memory pool location

Currently done at function level

Future work: smaller scopes

PEBIL is useful for practical memory address stream collectionMessage passing or threaded

https://github.com/mlaurenzano/PEBIL

michaell@sdsc.edu

Questions?