Download Presentation - The PPT/PDF document "Parallel Processing Sharing the load" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation on theme: "Parallel Processing Sharing the load"— Presentation transcript:
Slide1
Parallel Processing
Sharing the load
Slide2
Inside a Processor
Chip in Package
Circuits
Primarily Crystalline Silicon
1 mm – 25 mm on a side
100 million to billions of transistors
current “feature size”
(process)
~ 14 nanometers
Package provides:
communication with motherboard
heat dissipation
Slide3
Moore's LawNumber of transistors in same area doubles every 2 years
Net effects:
Processing power doubles approximately every 18 months
Slide4
Exponential GrowthDoubling is exponential growth
Year
0
1.5
3
4.567.59
10.5
12
Speed
1
2
4
8
16
32
64
128
256
Slide5
Moore's LawIf Moore's Law applied to airplanes:
New York to Paris in 1978 $900 & 7 hours
Slide6
Moore's LawIf Moore's Law applied to airplanes:
New York to Paris in 1978 $900 & 7 hours
Now should be
$0.01 & < 1 second
Slide7
Slide8
Power Density Prediction circa 2000
4004
8008
8080
8085
8086
286
386
486
Pentium® proc
P6
1
10
100
1000
10000
1970
1980
1990
2000
2010
Year
Power Density (W/cm2)
Hot Plate
Nuclear Reactor
Rocket Nozzle
Source: S. Borkar (Intel)
Sun’s Surface
Core 2
Slide9
Going Multi-core Helps Energy Efficiency
Speed takes power,
Power = heat
Can run at 80% speed with 50% power
Slide10
MultiCore
Multicore : Multiple processing
cores
on one chip
Each core can run a different program
Slide11
Moore's Law Related Curves
Slide12
Moore's Law Related Curves
Slide13
IssuesNot every part of a problem scales well
Parallel : can run at same time
Serial : must run one at a time in order
Slide14
5 workers can do parallel portion in 1/5
th
the time
Can't affect serial part
Speedup Issues
TimeNumber of CoresParallel portionSerial portion1
5
Slide15
Speedup Issues
Time
Number of Cores
Parallel portion
Serial portion
1
2
3
4
5
Increasing workers provide diminishing returns
Slide16
Amdahl’s Law
Amdahl’s law :
Predicts how many times faster
N workers can do a task in which
P portion is parallel
Slide17
Amdahl’s Law
60% of a job can be made parallel. We use 2 processors:
1.43x faster with 2 than 1
Slide18
Amdahl’s Law
60% of a job can be made parallel. We use 3 processors:
1.67x faster than with 1 worker
Slide19
Amdahl’s Law
Always have to do 40% of the work in serial
With infinite workers:
Slide20
Amdahl’s Law
Always have to do 40% of the work in serial
With infinite workers:
Only 2.5x faster!
2.5
Slide21
LimitsMax speedup limited by parallel portion of code:
Slide22
Speedup Issues : Overhead
Even assuming no sequential portion, there’s…
Time to think how to
divide the problem up Time to
hand out small “work units” to workers All workers may not work equally fastSome workers may fail
There may be contention for shared resources Workers could overwriting each others’ answersYou may have to wait until the last worker returns to proceed (the slowest / weakest link problem)There’s time to merge the results back together
