Parallel Processing Sharing the load
79K - views

Parallel Processing Sharing the load

Similar presentations


Download Presentation

Parallel Processing Sharing the load




Download Presentation - The PPT/PDF document "Parallel Processing Sharing the load" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentation on theme: "Parallel Processing Sharing the load"— Presentation transcript:

Slide1

Parallel Processing

Sharing the load

Slide2

Inside a Processor

Chip in Package

Circuits

Primarily Crystalline Silicon

1 mm – 25 mm on a side

100 million to billions of transistors

current “feature size”

(process)

~ 14 nanometers

Package provides:

communication with motherboard

heat dissipation

Slide3

Moore's LawNumber of transistors in same area doubles every 2 years

Net effects:

Processing power doubles approximately every 18 months

Slide4

Exponential GrowthDoubling is exponential growth

Year

0

1.5

3

4.567.59

10.5

12

Speed

1

2

4

8

16

32

64

128

256

Slide5

Moore's LawIf Moore's Law applied to airplanes:

New York to Paris in 1978 $900 & 7 hours

Slide6

Moore's LawIf Moore's Law applied to airplanes:

New York to Paris in 1978 $900 & 7 hours

Now should be

$0.01 & < 1 second

Slide7

Slide8

Power Density Prediction circa 2000

4004

8008

8080

8085

8086

286

386

486

Pentium® proc

P6

1

10

100

1000

10000

1970

1980

1990

2000

2010

Year

Power Density (W/cm2)

Hot Plate

Nuclear Reactor

Rocket Nozzle

Source: S. Borkar (Intel)

Sun’s Surface

Core 2

Slide9

Going Multi-core Helps Energy Efficiency

Speed takes power,

Power = heat

Can run at 80% speed with 50% power

Slide10

MultiCore

Multicore : Multiple processing

cores

on one chip

Each core can run a different program

Slide11

Moore's Law Related Curves

Slide12

Moore's Law Related Curves

Slide13

IssuesNot every part of a problem scales well

Parallel : can run at same time

Serial : must run one at a time in order

Slide14

5 workers can do parallel portion in 1/5

th

the time

Can't affect serial part

Speedup Issues

TimeNumber of CoresParallel portionSerial portion1

5

Slide15

Speedup Issues

Time

Number of Cores

Parallel portion

Serial portion

1

2

3

4

5

Increasing workers provide diminishing returns

Slide16

Amdahl’s Law

Amdahl’s law :

Predicts how many times faster

N workers can do a task in which

P portion is parallel

Slide17

Amdahl’s Law

60% of a job can be made parallel. We use 2 processors:

1.43x faster with 2 than 1

Slide18

Amdahl’s Law

60% of a job can be made parallel. We use 3 processors:

1.67x faster than with 1 worker

Slide19

Amdahl’s Law

Always have to do 40% of the work in serial

With infinite workers:

 

Slide20

Amdahl’s Law

Always have to do 40% of the work in serial

With infinite workers:

Only 2.5x faster!

2.5

 

Slide21

LimitsMax speedup limited by parallel portion of code:

Slide22

Speedup Issues : Overhead

Even assuming no sequential portion, there’s…

Time to think how to

divide the problem up Time to

hand out small “work units” to workers All workers may not work equally fastSome workers may fail

There may be contention for shared resources Workers could overwriting each others’ answersYou may have to wait until the last worker returns to proceed (the slowest / weakest link problem)There’s time to merge the results back together

Slide23

Why Parallelism?

We have no choice!

Multicore processors are a plan B

Parallel processing takes new

Computer ArchitecturesAlgorithmsProgramming Tools