An Empirical Characterization of - PowerPoint Presentation

Slide1

An Empirical Characterization of Stream Programs and its Implications for Language and Compiler DesignBill Thies1 and Saman Amarasinghe21 Microsoft Research India2 Massachusetts Institute of Technology

PACT 2010Slide2

What Does it Take toEvaluate a New Language?

1000

0

2000

Lines of Code

Small studies make it hard to assess:

Experiences of new users over time

Common patterns across large programsSlide3

What Does it Take toEvaluate a New Language?Lines of CodeStreamIt

(PACT’10)

10K

0

20K

30KSlide4

What Does it Take toEvaluate a New Language?Lines of CodeStreamIt

(PACT’10)

10K

0

20K

30K

Our characterization:

- 65 programs

34,000 lines of code

Written by 22 students

Over period of 8 years

This allows:

Non-trivial benchmarks

Broad picture of application space

Understanding long-term user

experienceSlide5

Streaming Application DomainFor programs based on streams of dataAudio, video, DSP, networking, and cryptographic processing kernels Examples: HDTV editing, radar tracking, microphone arrays, cell phone base stations, graphics

Adder

Speaker

AtoD

FMDemod

LPF

1

Duplicate

RoundRobin

LPF

2

LPF

3

HPF

1

HPF

2

HPF

3Slide6

Streaming Application DomainFor programs based on streams of dataAudio, video, DSP, networking, and cryptographic processing kernels Examples: HDTV editing, radar tracking, microphone arrays, cell phone base stations, graphicsProperties of stream programsRegular and repeating computationIndependent filters with explicit communication

Adder

Speaker

AtoD

FMDemod

LPF

1

Duplicate

RoundRobin

LPF

2

LPF

3

HPF

1

HPF

2

HPF

3Slide7

StreamIt: A Language and Compilerfor Stream ProgramsKey idea: design language that enables static analysisGoals:Improve programmer productivity in the streaming domainExpose and exploit the parallelism in stream programsProject contributions:Language design for streaming [CC'02, CAN'02, PPoPP'05, IJPP'05]Automatic parallelization [ASPLOS'02, G.Hardware'05, ASPLOS'06, MIT’10]Domain-specific optimizations [PLDI'03, CASES'05, MM'08]Cache-aware scheduling

[LCTES'03, LCTES'05]Extracting streams from legacy code [MICRO'07]User + application studies [PLDI'05, P-PHEC'05, IPDPS'06]Slide8

StreamIt Language BasicsHigh-level, architecture-independent languageBackend support for uniprocessors, multicores (Raw, SMP), cluster of workstationsModel of computation: synchronous dataflowProgram is a graph of independent filtersFilters have an atomic execution stepwith known input / output ratesCompiler is responsible for scheduling and buffer managementExtensions to synchronous dataflow Dynamic I/O ratesSupport for sliding window operationsTeleport messaging [PPoPP’05]Decimate

Input

Output

1

10

1

1

x 10

x 1

x 1

[Lee &

Messerschmidt,

1987]Slide9

Statefulfloat->float filter LowPassFilter (int

N,

work peek

N

push

1

pop

1 { float result = 0;

for

(int i=0;

i

<

weights.length

;

i

++) {

result += weights[

i

] *

peek

(

i

);

}

push

(result);

pop

();

}

}

Example Filter: Low Pass Filter

N

filter

Stateless

float

[N]

weights;

) {

weights =

adaptChannel

();Slide10

Structured Streams

may be any StreamIt language construct

joiner

splitter

pipeline

feedback loop

joiner

splitter

splitjoin

filter

Each structure is single-input, single-output

Hierarchical and composableSlide11

StreamIt Benchmark Suite (1/2)Realistic applications (30):MPEG2 encoder / decoderGround Moving Target IndicatorMosaicMP3 subsetMedium Pulse Compression RadarJPEG decoder / transcoderFeature Aided TrackingHDTVH264 subsetSynthetic Aperture RadarGSM Decoder802.11a transmitteDES encryption

Serpent encryption

Vocoder

RayTracer

3GPP

physical layer

Radar

Array Front End

Freq-hopping

radio

Orthogonal Frequency Division MultiplexerChannel Vocoder

Filterbank

Target Detector

FM Radio

DToA

ConverterSlide12

StreamIt Benchmark Suite (2/2)Libraries / kernels (23):AutocorrelationCholeskyCRCDCT (1D / 2D, float / int)FFT (4 granularities)LatticeGraphics pipelines (4):Reference pipelinePhong shadingSorting routines (8)Bitonic sort (3 versions)Bubble SortComparison counting

Matrix Multiplication

Oversampler

Rate Convert

Time Delay Equalization

Trellis

VectAdd

Shadow volumes

Particle system

Insertion sort

Merge sort

Radix sortSlide13

3GPPSlide14

802.11aSlide15

Bitonic SortSlide16

DCTSlide17

FilterBankSlide18

GSM DecoderSlide19

MP3 Decoder SubsetSlide20

Radar Array FrontendSlide21

VocoderSlide22

Characterization OverviewFocus on architecture-independent featuresAvoid performance artifacts of the StreamIt compilerEstimate execution time statically (not perfect)Three categories of inquiry:Throughput bottlenecksScheduling characteristicsUtilization of StreamIt language featuresSlide23

Lessons Learned fromthe StreamIt LanguageWhat we did rightWhat we did wrongOpportunities for doing betterSlide24

1. Expose Task, Data, & Pipeline ParallelismSplitter

Joiner

Task

Data parallelism

Analogous to DOALL loops

Task parallelism

Pipeline parallelismSlide25

1. Expose Task, Data, & Pipeline ParallelismData parallelismTask parallelism

Pipeline parallelism

Splitter

Joiner

Splitter

Joiner

Task

Pipeline

Data

StatelessSlide26

1. Expose Task, Data, & Pipeline ParallelismData parallelism74% of benchmarks contain entirely data-parallel filtersIn other benchmarks, 5% to 96% (median 71%) of work is data-parallelTask parallelism82% of benchmarks containat least one

splitjoinMedian of 8

splitjoins

per benchmark

Pipeline parallelism

Splitter

Joiner

Splitter

Joiner

Task

Pipeline

DataSlide27

Characterizing Stateful Filters763 Filter Types49 Stateful Types

6%

Stateful

94%

Stateless

55%

Avoidable

State

45%

Algorithmic

State

27 Types with

“Avoidable State”

Due to

induction

variables

Sources of Algorithmic State

MPEG2:

bit-alignment, reference frame encoding, motion prediction, …

HDTV:

Pre-coding and

Ungerboeck

encoding

HDTV + Trellis:

Ungerboeck

decoding

GSM:

Feedback loops

Vocoder

:

Accumulator, adaptive filter, feedback loop

OFDM:

Incremental phase correction

Graphics pipelines:

persistent screen buffersSlide28

Characterizing Stateful Filters2. Eliminate Stateful Induction Variables

763 Filter Types

49

Stateful

Types

6%

Stateful

94%

Stateless

55%

Avoidable

State

45%

Algorithmic

State

27 Types with

“Avoidable State”

Due to

message

handlers

Due to

induction

variables

Due to

Granularity

Sources of Induction Variables

MPEG encoder:

counts frame # to assign picture type

MPD / Radar:

count position in logical vector for FIR

Trellis:

noise source flips every N items

MPEG encoder / MPD:

maintain logical 2D position (row/column)

MPD:

reset accumulator when counter overflows

Opportunity: Language primitive to return current iteration?Slide29

3. Expose Parallelism in Sliding WindowsLegacy codes obscure parallelism in sliding windowsIn von-Neumann languages, modulo functions or copy/shift operations prevent detection of parallelism in sliding windowsSliding windows are prevalent in our benchmark suite57% of realistic applications contain at least one sliding windowPrograms with sliding windows have 10 instances on averageWithout this parallelism, 11 of our benchmarks would have a new throughput bottleneck (work: 3% - 98%, median 8%)

0

1

2

3

4

5

6

7

8

9

10

11

input

output

FIR

0

1Slide30

Characterizing Sliding Windows44%FIR Filterspush 1pop 1peek N3GPP, OFDM, Filterbank, TargetDetect, DToA, Oversampler, RateConvert, Vocoder, ChannelVocoder, FMRadio

34 SlidingWindow Types

29%

One-item windows

pop N

peek N+1

Mosaic, HDTV,

FMRadio,JPEG decode / transcode, Vocoder

27%MiscellaneousMP3: reordering (peek >1000)802.11: error codes (peek 3-7)Vocoder / A.beam: skip data Channel Vocoder: sliding correlation (peek 100)Slide31

4. Expose Startup BehaviorsExample: difference encoder (JPEG, Vocoder)Required by 15 programs:For delay: MPD, HDTV, Vocoder, 3GPP, Filterbank, DToA, Lattice, Trellis, GSM, CRCFor picture reordering (MPEG)For initialization (MPD, HDTV, 802.11)For difference encoder or decoder: JPEG, Vocoder

Stateless

int

->

int

filter

Diff_Encoder() { int state = 0;

work push

1 pop 1

{

push

(

peek

(0) – state);

state =

pop

();

}

}

int

->

int

filter

Diff_Encoder

() {

prework

push

1

pop

1

{

push

(

peek

(0));

}

work push

1

pop

1

peek

2 {

push

(

peek

(1) –

peek

(0));

pop

();

}

}

StatefulSlide32

5. Surprise:Mis-Matched Data Rates UncommonThis is a driving application in many papersEg: [MBL94] [TZB99] [BB00] [BML95] [CBL01] [MB04] [KSB08]Due to large filter multiplicities, clever scheduling is needed to control code size, buffer size, and latencyBut are mis-matched rates common in practice? No!

1

2

3

2

7

8

7

5

x 147

x 98

x 28

x 32

CD-DAT

benchmark

multiplicities

Converts CD audio (44.1 kHz) to digital audio tape (48 kHz)Slide33

5. Surprise:Mis-Matched Data Rates UncommonExcerpt fromJPEG transcoder

Execute once

per steady stateSlide34

Characterizing Mis-Matched Data RatesIn our benchmark suite:89% of programs have a filter with a multiplicity of 1On average, 63% of filters share the same multiplicityFor 68% of benchmarks, the most common multiplicity is 1Implication for compiler design:Do not expect advanced buffering strategies to have a large impact on average programsExample: Karczmarek, Thies, & Amarasinghe, LCTES’03Space saved on CD-DAT: 14xSpace saved on other programs (median): 1.2xSlide35

A multi-phase filter divides its execution into many stepsFormally known a cyclo-static dataflowPossible benefits:Shorter latenciesMore natural codeWe implemented multi-phase filters, and we regretted itProgrammers did not understand the difference betweena phase of execution, and a normal function callCompiler was complicated by presences of phasesHowever, phases proved important for splitters / joinersRouting items needs to be done with minimal latencyOtherwise buffers grow large, and deadlock in one case (GSM)6. Surprise: Multi-Phase FiltersCause More Harm than Good

1

1

2

3

Step 1

F

F

Step 2Slide36

7. Programmers IntroduceUnnecessary State in FiltersProgrammers do not implement things how you expectOpportunity: add a “stateful” modifier to filter decl?Require programmer to be cognizant of the cost of statevoid->int

filter SquareWave

() {

int

x = 0;

work push

1 { push(x); x = 1 - x;

}

}void->int

filter

SquareWave

() {

work push

2

{

push

(0);

push

(1);

}

}

Stateless

StatefulSlide37

8. Leverage and Improve Upon Structured StreamsOverall, programmers found it useful and tractable to write programs using structured streamsSyntax is simple to write, easy to readHowever, structured streams areoccasionally unnaturalAnd, in rare cases, insufficientSlide38

8. Leverage and Improve Upon Structured StreamsOriginal:Structured:

Compiler recovers unstructured graph

using

synchronization removal

[Gordon 2010]Slide39

8. Leverage and Improve Upon Structured StreamsOriginal:Structured:Characterization:49% of benchmarks have an Identity nodeIn those benchmarks, Identities accountfor 3% to 86% (median 20%) of instances

Opportunity:Bypass capability (ala GOTO) for streamsSlide40

Related WorkBenchmark suites in von-Neumann languages often include stream programs, but lose high-level propertiesMediaBenchALPBenchBerkeley MM WorkloadBrook language includes 17K LOC benchmark suiteBrook disallows stateful filters; hence, more data parallelismAlso more focus on dynamic rates & flexible program behaviorOther stream languages lack benchmark characterizationStreamC / KernelCCgIn-depth analysis of 12 StreamIt “core” benchmarks published concurrently to this paper [Gordon 2010]

HandBench

MiBench

NetBench

Baker

SPUR

SPEC

PARSEC

Perfect Club

SpidleSlide41

ConclusionsFirst characterization of a streaming benchmark suitethat was written in a stream programming language65 programs; 22 programmers; 34 KLOCImplications for streaming languages and compilers:DO: expose task, data, and pipeline parallelismDO: expose parallelism in sliding windowsDO: expose startup behaviorsDO NOT: optimize for unusual case of mis-matched I/O ratesDO NOT: bother with multi-phase filtersTRY: to prevent users from introducing unnecessary stateTRY: to leverage and improve upon structured streams

TRY: to prevent induction variables from serializing filtersExercise care in generalizing results beyond StreamItSlide42

Acknowledgments:Authors of the StreamIt BenchmarksSitij AgrawalBasier AzizJiawen ChenMatthew DrakeShirley FungMichael GordonOla JohnssonAndrew LambChris LegerMichal KarczmarekDavid Maze

Ali

Meli

Mani Narayanan

Satish

Ramaswamy

Rodric

RabbahJanis SermulinsMagnus

Stenemo

Jinwoo SuhZain ul-Abdin

Amy Williams

Jeremy Wong