PinADX : Customizable Debugging with Dynamic Instrumentation - PowerPoint Presentation

Slide1

PinADX: Customizable Debugging with Dynamic Instrumentation

Gregory Lueck, Harish Patil, Cristiano PereiraIntel CorporationCGO 2012, San Jose, USA

1

Slide2

Hypothetical Problem 1

2(gdb

) run

Program

received signal SIGSEGV, Segmentation fault.

0x0000000000000000 in ?? ()(gdb) bt#0 0x0000000000000000 in ?? ()#1 0x0000000000000000 in ?? ()

Crash with bad PC and no stack trace

Corrupted return address someplace ...

Want to stop BEFORE bad “ret” instruction

Slide3

Hypothetical Problem 2

3

thread stack

?

thread stack

?

thread stack

?

thread stack

?

...

Massively threaded application

How much stack space needed?

At what point does each thread use its max stack?

Slide4

Traditional Debugger Breakpoints?

4

Original

Application

…

Debugger catches trap

Check if “ret” is to good PC

If yes, resume

How can debugger find all “ret” instructions?

Horribly slow to trap on each return

Foo:

ret

Bar:

…

ret

Application

In Debugger

…

Foo:

trap

Bar:

…

trap

Overwrite each “ret” with trap

ret

ret

Slide5

Dynamic Binary Instrumenation (DBI)

5

Application

i

f (return to bad PC)

Breakpoint()

Application

i

f (stack too big)

Breakpoint()

Much faster – avoids trap overhead

DBI can find all “ret” instructions reliably

General

approach – solves stack problem (and others)

BUT difficult to integrate with debugger

sub 0x60, %

sp

…

Foo:

ret

Bar:

…

ret

Foo:

Bar:

sub 0x10, %

sp

…

…

Instrumentation

Slide6

Pin Overview

Tool

JIT compiler

Application

Code Cache

T

races

Tool controls instrumentation

(e.g. “if return to bad PC”)

JIT compiler fetches application instructions, calls tool to instrument

Instrumented instructions stored in code cache for efficiency

Fetch

Instrument

Optimize

Store &

execute

6

Slide7

Original Code

Code Cache

JIT Compiler Overview

1’

2

1

6

3

5

2’

6

’

Pin

4

3

’

5

’

Tool inserts instrumentation (e.g. check if return to bad PC)

Dynamic recompilation makes debugging hard

7

Slide8

Process

running under Pin

PinADX Architecture

8

Tool

Pin

Application

Debugger

PinADX presents “pure” view of application. Hides effect of instrumentation and recompilation.

Tool extends debugger via instrumentation.

PinADX

core

GDB

or

Microsoft Visual Studio 11

Supports Linux & Windows

Slide9

Rest of the Talk

Introduction / MotivationExample: Using “Stack Debugger” extensionExample: Authoring “Stack Debugger” extensionImplementing PinADX9

Slide10

$ gdb ./my-application

(gdb) target remote :1234

(gdb) break PrintHello

Breakpoint 1 at 0x4004dd: file

hw.c

, line 13

(

gdb

)

cont

Breakpoint 1, PrintHello () at hw.c:13

(gdb) backtrace

#0

PrintHello

() at hw.c:13

#1 main () at hw.c:7

(gdb) x/2i $pc

=> 0x4004dd <PrintHello+4>: mov $0x4005e8,%edi

0x4004e2 <PrintHello+9>: callq

0x4003b8

Example – Stack Debugger

10

$ pin –appdebug –t stack-debugger.so -- ./my-application

Application stopped until continued from debugger.

Start GDB, then issue this command at the (

gdb

) prompt:

target remote :

1234

Run application under Pin

Debugger connected

to Pin

(

gdb

)

cont

Breakpoint 1, PrintHello () at hw.c:13

(gdb) backtrace

#0

PrintHello

() at hw.c:13

#1 main () at

hw.c:7

(gdb) x/2i $pc

=> 0x4004dd <PrintHello+4>: mov $0x4005e8,%edi

0x4004e2 <PrintHello+9>: callq 0x4003b8

(

gdb

) break

PrintHello

Breakpoint 1 at 0x4004dd: file

hw.c

, line

13

Slide11

(

gdb) monitor stackbreak 4000

Break when thread uses 4000 stack bytes

(gdb) cont

Stopped: Thread uses 4004 bytes of

stack

(gdb) backtrace

#0 0x3f07214445

in _dl_runtime_resolve

()

#1 0x00004004e7

in PrintHello () at hw.c:13

#2 0x00004004d2

in main () at hw.c:7

(gdb) monitor stackbreak 10000

Break when thread uses

10000

stack bytes

(gdb) break exit

Breakpoint 2 at

0x7fffe60f9650

(gdb) contBreakpoint 2,

0x7fffe60f9650 in exit ()(gdb) monitor stats

Maximum stack usage: 8560 bytes.Example – Stack Debugger

11

Stop when application uses too much stack

(gdb) monitor stackbreak 4000

Break when thread uses 4000 stack bytes

(gdb) cont

Stopped: Thread uses 4004 bytes of stack

(gdb) monitor stackbreak 10000

Break when thread uses 10000 stack bytes

(gdb) break exit

Breakpoint 2 at 0x7fffe60f9650

(gdb) cont

Breakpoint 2, 0x7fffe60f9650 in exit ()

(gdb) backtrace

#0 0x3f07214445 in _dl_runtime_resolve ()

#1 0x00004004e7 in PrintHello () at hw.c:13

#2 0x00004004d2 in main () at hw.c:7

(gdb) monitor stats

Maximum stack usage: 8560 bytes.

Slide12

Rest of the Talk

Introduction / MotivationExample: Using “Stack Debugger” extensionExample: Authoring “Stack Debugger” extensionImplementing PinADX12

Slide13

Stack Debugger – Instrumentation

13

Thread Start:

[…]

sub $0x60, %

esp

cmp

%

esi

, %

edx

jle

<L1>

size =

StackBase

- %

esp

;

if (size >

MaxStack

)

MaxStack

= size;

if (size >

StackLimit

)

TriggerBreakpoint

();

StackBase

= %

esp

;

MaxStack

= 0;

After each stack-changing instruction

Record initial stack

Slide14

VOID

Instruction(INS

ins

, VOID *)

{

if (

INS_RegWContain

(ins, REG_STACK_PTR))

{

IPOINT where = (

INS_HasFallThrough

(ins)) ?

IPOINT_AFTER : IPOINT_TAKEN_BRANCH;

INS_InsertCall

(ins, where, (AFUNPTR)

OnStackChange

,

IARG_REG_VALUE, REG_STACK_PTR,

IARG_THREAD_ID, IARG_CONTEXT, IARG_END);

}

}

VOID

OnStackChange

(ADDRINT

sp, THREADID tid, CONTEXT *

ctxt

)

{

size_t

size =

StackBase

-

sp

;

if (size >

StackMax

)

StackMax

= size;

if (size >

StackLimit

) {

ostringstream

os

;

os

<< "Stopped: Thread

uses "

<<

size << " stack bytes.";

PIN_ApplicationBreakpoint

(

ctxt

,

tid

, FALSE,

os.str

());

}

}

Stack Debugger – Implementation

14

Instrumentation

Analysis

Instrument only instructions that change $SP

Call after each instruction

Slide15

int

main() {

[…]

PIN_AddDebugInterpreter

(

HandleDebugCommand

, 0);

}

BOOL

HandleDebugCommand

(

const

string &

cmd

, string *result) {

if (

cmd

== "stats")

{

ostringstream os;

os << "Maximum stack usage: " << StackMax

<< " bytes.\n"; *result = os.str();

return TRUE; }

else if (cmd.find("stackbreak

") == 0)

{

StackLimit

= /* parse limit */;

ostringstream

os

;

os

<<

"Break when thread

uses " << limit << " stack bytes.";

*result =

os.str

();

return TRUE;

}

return FALSE; // Unknown command

}

Stack Debugger – Implementation

15

Slide16

Visual Studio IDE Extension

16

Slide17

Other Debugger Extensions

Intel Inspector XE ProductMemory CheckerThread CheckerIntel SDE: Instruction emulationDebug from log file (PinPlay, CGO 2010)

Dynamic slicing (Rajiv Gupta, UC Riverside)

Cmp$im: Cache simulator

Write your own!

17

Slide18

Rest of the Talk

Introduction / MotivationExample: Using “Stack Debugger” extensionExample: Authoring “Stack Debugger” extensionImplementing PinADX18

Slide19

Process

running under Pin

PinADX Architecture

19

Tool

Pin

Application

Debugger

PinADX presents “pure” view of application. Hides effect of instrumentation and recompilation.

Tool extends debugger via instrumentation.

PinADX

core

GDB

or

Microsoft Visual Studio 11

Slide20

Communication Details

20

Very low level

Symbol processing in debugger

Expression evaluation in debugger

Extension of GDB’s remote debugging protocol

Commands

Read / write registers, memory

Set breakpoints

Continue, single-step, stop

Notifications

Breakpoint triggered

Caught signal

Application exited

Debugger

PinADX

core

Pin

Slide21

Communication Details

21

Breakpoint alternatives

Insert real INT3 trap instruction

Virtualize inside Pin VM

See paper for details

Commands

Read / write registers, memory

Set breakpoints

Continue, single-step, stop

Notifications

Breakpoint triggered

Caught signal

Application exited

Debugger

PinADX

core

Pin

Slide22

set breakpoint at 4

continue

Code Cache

Original Code

Breakpoint

1’

2

3

1

4

5

6

breakpoint notification

Execution stops in Pin

Waits for GDB to continue

BP

2’

3’

PinADX

core

Debugger

22

Slide23

Code Cache

Original Code

Single Step

1’

2

3

1

4

5

6

step complete notification

Debugger

Execution stops in Pin

Waits for GDB to continue

do single-step

PinADX

core

23

Slide24

Thanks

Mark Charney – SDE software emulatorAndria Pazarloglou – Created VS11 GUI pluginGregg Miskelly –

Microsoft VS11 debugger architect

Robert

Cohn –

Father of Pin24

Slide25

Summary

DBI can implement powerful debugger featuresAPI allows Pin tools to extend debugger easilyMulti-platformLinux: GDBWindows: Microsoft Visual Studio 11 (soon)

Works with off-the-shelf debuggers

25

http://pintool.org