Heap Growth Detection in C++ - PowerPoint Presentation

Heap Growth Detection in C++
Heap Growth Detection in C++

Heap Growth Detection in C++ - Description


GrowthTracker 1 Heap Growth Detection in C Motivation Scalable City needs to run continuously Many months without interventionaccess Had slow growth of memory Caused crash after several weeks ID: 512743 Download Presentation

Tags

memory growth size interval growth memory interval size tracker data tumor leak unbounded time amp false def detection aggregate

Embed / Share - Heap Growth Detection in C++


Presentation on theme: "Heap Growth Detection in C++"— Presentation transcript


Slide1

Heap Growth Detection in C++GrowthTracker

1Slide2

Heap Growth Detection in C++Motivation

Scalable City needs to run continuouslyMany months without intervention/accessHad slow growth of memoryCaused crash after several weeksAvailable analysis tools reported no leaks!

Software frees all memory correctly!Had different kind of undetected memory issue

2Slide3

What is a Memory Leak?

3Slide4

What is a Memory Leak?

Has become a broad term for memory mismanagementDefinition depends on the programming languageDoes it have Garbage Collection?

4Slide5

What is a Memory Leak?

C/C++ Programmers generally subscribe to traditional definition: Def. 1: A memory leak occurs iff all references to a region of memory are lost before the memory is freed.

5Slide6

What is a Memory Leak?

Java/C# Programmers generally have a different definition: Def. 2: A memory leak occurs when a program’s memory is unexpectedly growing.

Using Def. 1, Java/C# don’t have memory leaks

Garbage Collector frees memory that is no longer referenced

6Slide7

What is a Memory Leak?

7

Def .2

(unbounded growth)

Def .1

(

lost refs

)

C/C++

Java/C#

Def .2

(unbounded growth)

Def .2

(unbounded growth)

Def .1

(

lost refs

)Slide8

What is a Memory Leak?

We need new terminology!Java/C# programmers Concerned with unbounded memory growth, not leaksReferences exist to all memory growthHave clearer understanding of their problem

C/C++ programmersMultiple definitions causes confusionFalse

sense of security when tools report no memory leaks

Unbounded memory growth may still exist!

A term for unbounded memory growth not caused by memory leaks, would clearly parse the 2 problems

8Slide9

New “Leak” Terminology

Properties of new termUnbounded Heap GrowthReference(s) retainedManifests as dynamically growing data structureWill eventually kill the processDoesn’t fit leaky pipe metaphor

Memory TumorStructure of cells that exhibits unbounded growth

9

dataSlide10

Memory Tumor Example

void main(){ queue q; while( key

!= ESC ) // exits when ESC key pressed

q.push

(0);}

q will

grow constantly while the program

runs

there

are no memory

leaks

if

the user hits ESC, all memory is freed when

q

goes out of

scope

10Slide11

Leak vs. Tumor

11

X

X

X

X

X

Leak

Tumor

heap

heapSlide12

Separation of Concerns

12

Memory Leak

(

Def

1: lost refs

)

C/C++/Java/C#

Memory TumorSlide13

Current C++ Tumor Detection Tools

SWAT and HoundOnly ones we’re aware ofBoth use Staleness detectionMisses some tumors by designMemory can be accessed, but not neededInvestigate at allocation levelDon’t need to modify source code

Not open source or commercially available

13Slide14

Memory Tumor Detection Theory

Tumors Data Structures that grow without boundHealthy data structuresWill growMaximum size stabilizes

14Slide15

Memory Tumor Detection Challenges

Detect all growth that doesn’t stabilize Don’t dismiss non-stale growthTests must exhibit the growth that exists in a program’s implementationSupport Multithreaded programs

15Slide16

Memory Tumor Detection Approach

Growth Tracker ToolContainer TrackingKeep references to all data structures in memoryGrowth TrackingTrack data structure size changes over timeIdentify those with unbounded growthAutomated Test

Cyclically execute all code paths (user created)

16Slide17

Growth Tracker Tool

Container trackingCAT (Central Aggregate Tracker)Maintains references to all aggregates in the systemCreate wrappers for each aggregate type in system

Templated constructors, multiple inheritanceAdd to CAT on construction, remove on destructionNamespace replacement to enable wrappers

Find and replace to apply new namespace

Wrappers disabled with compile time flag

Example:

trak

::std::vector<

int

>

17Slide18

Growth Tracker Tool

Growth TrackingTake periodic samples of the CATExponentially increasing interval sizesReduces false positives & negatives over timeReport growing aggregates at each sample

18Slide19

Growth Tracking Heuristic

Take periodic samples of the CATTwo Interval Analysis1st interval establishes aggregate age, gives time to stabilize

2nd interval proves stability, non-tumors shouldn’t grow

2

nd

interval becomes the 1st

for next more accurate test

Exponentially increasing interval sizes

Reduces false positives & negatives over time

Monitor size maximums

Reduces size fluctuation false positives

At each interval report all aggregates that:

Increased their size maximum

Have existed for two full intervals

Prioritize results by size & reporting frequency

19Slide20

Growth Tracking Heuristic

Two interval analysis

20

time

memory

1

2

3

4

Not reported

(growth stabilized)

A Data Structure Memory FootprintSlide21

Diagnosing Unbounded Heap Growth in C++

Detection Approach

Two interval analysis

21

time

memory

1

2

3

4

Reported as tumor

(false positive)

Not reported

(growth stabilized)Slide22

Growth Tracking Heuristic

Take periodic samples of the CATTwo Interval Analysis

1st

interval establishes aggregate age, gives time to stabilize

2

nd

interval proves stability, non-tumors shouldn’t grow

2

nd

interval becomes the 1

st

for next more accurate test

Exponentially increasing interval sizes

Reduces false positives & negatives over time

Monitor size maximums

Reduces size fluctuation false positives

At each interval report all aggregates that:

Increased their size maximum

Have existed for two full intervals

Prioritize results by size & reporting frequency

22Slide23

Diagnosing Unbounded Heap Growth in C++

Detection Approach

Growth TrackingExponentially increasing interval size

23

time

memory

1

2

3

4

In this example: constant intervals would not report growth half the timeSlide24

Growth Tracking Heuristic

Take periodic samples of the CATTwo Interval Analysis

1st

interval establishes aggregate age, gives time to stabilize

2

nd

interval proves stability, non-tumors shouldn’t grow

2

nd

interval becomes the 1

st

for next more accurate test

Exponentially increasing interval sizes

Reduces false positives & negatives over time

Monitor size maximums

Reduces size fluctuation false positives

At each interval report all aggregates that:

Increased their size maximum

Have existed for two full intervals

Prioritize results by size & reporting frequency

24Slide25

Diagnosing Unbounded Heap Growth in C++ Detection Approach

Growth TrackingMax size variable

25

1

2

3

time

memory

ceiling

4

Growth would be reported without max sizeSlide26

Growth Tracking Heuristic

Take periodic samples of the CATTwo Interval Analysis

1st

interval establishes aggregate age, gives time to stabilize

2

nd

interval proves stability, non-tumors shouldn’t grow

2

nd

interval becomes the 1

st

for next more accurate test

Exponentially increasing interval sizes

Reduces false positives & negatives over time

Monitor size maximums

Reduces size fluctuation false positives

At each interval report all aggregates that:

Increased their size maximum

Have existed for two full intervals

Prioritize results by size & reporting frequency

26Slide27

Growth Tracker Targets

Multi-threaded ApplicationsInitial CAT implementation worksRequires locking for each aggregate constructorPotential to diminish multi-threaded performanceGood starting point

Need new CAT implementationEliminate LocksMultiple bucket approach Map aggregate construction from different threads to buckets

Design can accelerate sampling process as well

27Slide28

Growth Tracker Drawbacks

Source code modificationTracking requires compilation with our wrappersAllows consideration of Objects not just allocations.Limited information about identified tumors

Full type string & allocation numberCode location possiblerequires stack tracing (slower)Reliance on the user

Must identify custom data structures

Must run feature complete and cyclic test

28Slide29

Growth Tracker Drawbacks

Multi-threaded potential slow downPersistent bucketsExample: Linear hash table with std::vector bucketsMore useful to include child bucket sizes in parent’s output and stop reporting individual children

Multiple instances of same tumor reportedParent report including children would resolve

29Slide30

Growth Tracker Results

Scalable CityIdentified tumorEliminated memory growthOgre3D Rendering EngineIdentified 2 tumors

Our fix integrated into their code baseBullet Physics EngineTests revealed no tumors in Core1 tumor found in demo framework

30Slide31

Growth Tracker Results

Google Chrome / ChromiumIdentified 21 tumorsFixed the fastest growing tumor ourselvesWebKit (Safari Browser, etc.)

Identified 2 tumorsSubmitted fix to code base

31Slide32

Growth Tracker Paper

Recently accepted for publicationIEEE International Conference on Software Testing, Verification and Validation (ICST 2013)

32Slide33

Growth Tracker Proposed Work

Resolve Multithreaded locking limitationsSolution designed, needs implementationReduce tracking of temporaries

Detect stack-based data structuresMulti-layer CAT to separate entries by ageWill reduce overhead of CAT insertion/removal

33Slide34

Growth Tracker Proposed Work

AutomationReduce reliance on the userDetect custom data structures

Automatically create wrappers when possibleImprovements to code transformation processAfter initial code conversion, detect when wrapper is forgotten.

34Slide35

Growth Tracker Proposed Work

Prioritize tracking parent data structuresWould address persistent bucket problemWould reduce reports of multiple instances of same tumor

Must identify relationships between data structures.

35

Shom More....
By: sherrill-nordquist
Views: 53
Type: Public

Download Section

Please download the presentation from below link :


Download Presentation - The PPT/PDF document "Heap Growth Detection in C++" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Try DocSlides online tool for compressing your PDF Files Try Now