Inside a Processor. Chip in Package. Circuits. Primarily Crystalline Silicon. 1 mm – 25 mm on a side. 100 million to billions of transistors. current “feature size” . (process). ~ 14 nanometers. ID: 756809
DownloadNote - The PPT/PDF document "Parallel Processing Sharing the load" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Parallel Processing
Sharing the load
Slide2Inside a Processor
Chip in Package
Circuits
Primarily Crystalline Silicon
1 mm – 25 mm on a side
100 million to billions of transistors
current “feature size”
(process)
~ 14 nanometers
Package provides:
communication with motherboard
heat dissipation
Slide3Moore's LawNumber of transistors in same area doubles every 2 years
Net effects:
Processing power doubles approximately every 18 months
Slide4Exponential GrowthDoubling is exponential growth
Year
0
1.5
3
4.567.59
10.5
12
Speed
1
2
4
8
16
32
64
128
256
Slide5Moore's LawIf Moore's Law applied to airplanes:
New York to Paris in 1978 $900 & 7 hours
Slide6Moore's LawIf Moore's Law applied to airplanes:
New York to Paris in 1978 $900 & 7 hours
Now should be
$0.01 & < 1 second
Slide7Slide8
Power Density Prediction circa 2000
4004
8008
8080
8085
8086
286
386
486
Pentium® proc
P6
1
10
100
1000
10000
1970
1980
1990
2000
2010
Year
Power Density (W/cm2)
Hot Plate
Nuclear Reactor
Rocket Nozzle
Source: S. Borkar (Intel)
Sun’s Surface
Core 2
Slide9Going Multi-core Helps Energy Efficiency
Speed takes power,
Power = heat
Can run at 80% speed with 50% power
Slide10MultiCore
Multicore : Multiple processing
cores
on one chip
Each core can run a different program
Slide11Moore's Law Related Curves
Slide12Moore's Law Related Curves
Slide13
IssuesNot every part of a problem scales well
Parallel : can run at same time
Serial : must run one at a time in order
Slide145 workers can do parallel portion in 1/5
th
the time
Can't affect serial part
Speedup Issues
TimeNumber of CoresParallel portionSerial portion1
5
Slide15Speedup Issues
Time
Number of Cores
Parallel portion
Serial portion
1
2
3
4
5
Increasing workers provide diminishing returns
Slide16Amdahl’s Law
Amdahl’s law :
Predicts how many times faster
N workers can do a task in which
P portion is parallel
Slide17Amdahl’s Law
60% of a job can be made parallel. We use 2 processors:
1.43x faster with 2 than 1
Slide18Amdahl’s Law
60% of a job can be made parallel. We use 3 processors:
1.67x faster than with 1 worker
Slide19Amdahl’s Law
Always have to do 40% of the work in serial
With infinite workers:
Slide20
Amdahl’s Law
Always have to do 40% of the work in serial
With infinite workers:
Only 2.5x faster!
2.5
Slide21
LimitsMax speedup limited by parallel portion of code:
Slide22Speedup Issues : Overhead
Even assuming no sequential portion, there’s…
Time to think how to
divide the problem up Time to
hand out small “work units” to workers All workers may not work equally fastSome workers may fail
There may be contention for shared resources Workers could overwriting each others’ answersYou may have to wait until the last worker returns to proceed (the slowest / weakest link problem)There’s time to merge the results back together
Slide23Why Parallelism?
We have no choice!
Multicore processors are a plan B
Parallel processing takes new
Computer ArchitecturesAlgorithmsProgramming Tools
Next Slides
Parallel image processing
Parallel processing, flynn’s classification of computers
Tight bounds for
Introduction to parallel processing
Prepared 6/4/2011 by t. o’neil for 3460:677, fall 2011, t
Inspector-executor load balancing algorithms for block-spar
Chapter pipeline and vector processing section
Intro to gpu’s for parallel computing
Dynamic load sharing and balancing