2 Turing machine RAM Figure Logic circuit model RAM Random Access Machine Operations supposed to be executed in one unit time 1 Control operations such as ID: 613134
Download Presentation The PPT/PDF document "1 Lecture 2: Parallel computational mode..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
1
Lecture 2: Parallel computational modelsSlide2
2
Turing machine
RAM (Figure ) Logic circuit model RAM (Random Access Machine)Operations supposed to be executed in one unit time(1) Control operations such as if，goto（for and while can be realized by for and goto. ）(2) I/O operations such as print (3) Substitution operations such as a = b (4) Arithmetic and logic operations such as +, -, AND.
Control Unit (contains algorithms)
CPU
Memory
(unlimited size)
Data
Sequential computational modelsSlide3
3
O-notation for computing complexity
Definition
Assume that f(n) is a positive function. If there are two positive constants c, n0 such that f(n) ≦ c g(n) for all n ≧ n0, then we say f(n) = O( g(n) ). For example, 3n2-5n = O(n2) n log n + n = O(n log n) 45 = O(1)（The item which grows most quickly)Slide4
4
Sequential algorithms Parallel algorithmsModels RAM Many typesData division Not necessary Most importantAnalysis Computing time Computing time Memory size Communicating time Number of processors
Algorithm analysis for sequential and
p
arallel algorithmsSlide5
5
PRAM (Parallel RAM) model
PRAM
consists of a number of RAM (Random Access Machine) and a shared memory. Each RAM has a unique processor number. Processors act synchronously. Processor execute the same program. （According to the condition fork based on processor numbers, it is possible to executed different operations.) Data communication between processors (RAMs) are held through the shared common memory. Each processor can write data to and read data from one memory cell in O(1) time.
Shared
Memory
RAM 1
RAM 2
RAM mSlide6
6
Features of PRAM
Merits
Parallelism of problems can be considered essentially. Algorithms an be easily designed. Algorithms can be changed easily to ones on other parallel computational models.Demerits Communicational cost is not considered. (It is not realistic that one synchronized reading and writing can be done in one unit time.) Distributed memory is not considered. In the following, We use PRAM to discuss parallel algorithms. Slide7
7
Analysis of parallel
algorithms on PRAM model
Computing time T(n) Number of processors P(n) Cost P(n) × T(n) Speed-up Ts(n)/T(n) Ts(n): Computing time of the optimal sequential algorithm）Cost optimal parallel algorithms The cost is the same as the computing time of the optimal sequential algorithm, i.e., speed-up is the same as the number of processors.2. Time optimal parallel algorithms Fastest when using polynomial number of processors. 3. Optimal parallel algorithms Cost and time optimal.Slide8
8
World of sequential computation
P
problems：the class of problems which can be solved in polynomial time (O(n )).NP problems：the class of problems which can be solved non-determinately in polynomial time. NP-complete problems： the class of NP problems which can be reduced to each other. P = NP ?World of parallel computationNC Problems: the class of problems which can be solved in log-polynomial time (O(lg n) ). P-complete problems：the class of problems which are not NC problems and can be reduced to each other. Similarly, NC = P ?NC-class and P-classkkAnalysis of parallel algorithms on PRAM model Slide9
9
An Example of PRAM Algorithms
Problem
：Find the sum of n integers (x1, x2, ... , xn) - Assume that n integers are put in array A[1..n] on the shared memory. - To simplify the problem, let n = 2k (k is an integer). main () { for (h=1; h≦log n; h++) { if (index of processor i ≦ n/2h
) processor i do
{
a
= A[2i-1]; /* Reading from the shared memory*/
b = A[2i];
/* Reading from the shared memory*/ c = a + b;
A[
i
] = c; /* Writing to the shared memory */
} } if (the number of processor == 1) printf
("%d¥n", c
);
}Slide10
10
Processor
Pi reads A[2i-1], A[2i] from the shared memory, then writes their summation to A[
i] of the shared memory.PiA[i]A[2i-1]A[2i]
A[1]
P
1
A[2]
P
2
A[3]
A[4]
A[n]
An Example of PRAM
A
lgorithms Slide11
11
Find
the summation of 8 integers (x
1, x2, ... , x8). Sequential algorithmP1P
１
P
2
P
3
P
4
P
１
P
２
x
1
x
2
x
3
x
4
x
5
x
6
x
7
x
8
Output
Input
Step 1
Step 2
Step 3
Parallel algorithm
An Example of PRAM
A
lgorithms Slide12
12
Analysis
of the algorithm
Computing time：for loop is repeated log n times, each loop can be executed in O(1) time → O(log n) time Number of processors：not larger than n/2 → n/2 processors Cost: O(n log n)It is not cost optimal since the optimal sequential algorithmrun in Θ(n) time.An Example of PRAM Algorithms Slide13
13
EREW (Exclusive read exclusive write) PRAM
Both concurrent reading and concurrent writing are prohibited. CREW (Concurrent Read Exclusive write) PRAM Concurrent reading is allowed, but concurrent writing is prohibited. Classification of PRAM by the access restriction Slide14
14
CRCW (Concurrent Read Concurrent write) PRAM
Both concurrent reading and concurrent writing are allowed. It is classified furthermore: - common CRCW PRAM Concurrent writing happens is only if the writing data are the same. - arbitrary CRCW PRAM An arbitrary data is written. - priority CRCW PRAM The processor with the smallest number writes the data. Classification of PRAM by the access restriction Slide15
15
Algorithms on different PRAM models
Algorithms for calculating
and of n bits (Input is put in array A[1..n]) Algorithm on EREW PRAM modelmain (){ for (h=1; h≦log n; h++) { if (index of processor i ≦ n/2h) processor i do { a = A[2i-1]; b = A[2i]; if ((a==1) and (b==1)) a[i] = 1; }}} Algorithm on common CRCW PRAM modelmain (){ if (A[index of processor i
] == 1) processor i do
A[1] = 1;}
O(log n) time
n/2 processors
O(1) time
n processors
Abilities of PRAM models: EREW < CREW < CRCWSlide16
Exercise
1. Suppose
n×n
matrix A and matrix B are saved in two dimension arrays. Design a PRAM algorithm for A+B using n and n×n processors, respectively. Answer the following questions: What is the PRAM model that you use in your algorithm?What is the running time? Is your algorithm cost optimal? Is your algorithm time optimal? 2. Design a PRAM algorithm for A+B using n processors. Answer the same questions. 3. Design a PRAM algorithm for A+B using k (k ≤n×n) processors. Answer the same questions. 16