Lecture Multiprocessors Topics multiprocessor intro and taxonomy symmetric sharedmemory multiprocessors Sections

Presentation on theme: "Lecture Multiprocessors Topics multiprocessor intro and taxonomy symmetric sharedmemory multiprocessors Sections"— Presentation transcript:

1 Lecture 17: Multiprocessors shared-memory multiprocessors (Sections 4.1-4.2) 2 Taxonomy •SISD: single instruction and single data stream: uniprocessor•MISD: no commercial multiprocessor: imagine data goingthrough a pipeline of execution engines•SIMD: vector architectures: lower flexibility•MIMD: most multiprocessors today: easy to construct withoff-the-shelf computers, most flexibility 3 Memory Organization - I memory – since all processors see the same memorybottleneck? – not if you have large caches and employ 4 SMPs or Centralized Shared-Memory Processor Caches Processor Caches Processor Caches Processor Caches Main Memory I/O System 5 Memory Organization - II •For higher scalability, memory is distributed amongprocessors distributed memory multiprocessors•If one processor can directly address the memory localto another processor, the address space is shared distributed shared-memory (DSM) multiprocessor•If memories are strictly local, we need messages tocommunicate data cluster of computers or multicomputers•Non-uniform memory architecture (NUMA) since localmemory has lower latency than remote memory 6 Distributed Memory Multiprocessors Processor& Caches Memory I/O Processor& Caches Memory I/O Processor& Caches Memory I/O Processor& Caches Memory I/O Interconnection network 7 Shared-Memory Vs. Message-Passing Shared-memory:•Well-understood programming model•Communication is implicit and hardware handles protection•Hardware-controlled cachingMessage-passing:No cache coherence simpler hardware•Explicit communication easier for the programmer torestructure code•Sender can initiate data transfer 8 Ocean Kernel for i &#x/MCI; 10;&#x 000;&#x/MCI; 10;&#x 000;A[i,j] Å0.2 * (A[i,j] + neighbors);&#x/MCI; 11;&#x 000;&#x/MCI; 11;&#x 000;diff += abs(A[i,j] – temp); 9 Shared Address Space Model int n, nprocs;float **A, diff;LOCKDEC(diff_lock);BARDEC(bar1);main()beginread(n); read(nprocs);G_MALLOC();initialize (A);CREATE (nprocs,Solve,A);WAIT_FOR_END (nprocs);end mainprocedure Solve(A)int i, j, pid, done=0;float temp, mydiff=0;int mymin = 1 + (pid * n/procs);while (!done) domydiff = diff = 0;BARRIER(bar1,nprocs);for i mymin to mymaxfor j 1 to n do…endforendforLOCK(diff_lock);diff += mydiff;UNLOCK(diff_lock);BARRIER (bar1, nprocs);if (diff )endwhile 10 Message Passing Model )&#x/MCI; 16;&#x 000;&#x/MCI; 16;&#x 000;if (pid != nprocs-1)&#x/MCI; 17;&#x 000;&#x/MCI; 17;&#x 000;SEND(&myA[nn,0], n, pid+1, ROW);&#x/MCI; 18;&#x 000;&#x/MCI; 18;&#x 000;if (pid != 0)&#x/MCI; 19;&#x 000;&#x/MCI; 19;&#x 000;RECEIVE(&myA[0,0], n, pid-1, ROW); 11 SMPs Value of X in Cache-A Cache-B Memory 1 - 1 1 1 0 12 Cache Coherence processors – write serialization 13 Cache Coherence Protocols status of that block – all cache controllers monitor the 14 Design Issues •Invalidate•Find data•Writeback / writethrough Processor Caches Processor Caches Processor Caches Processor Caches Main Memory I/O System •Cache block states•Contention for tags•Enforcing write serialization 15 Example Protocol RequestSourceBlock stateActionRead hitProcShared/exclRead data in cacheRead missProcInvalidPlace read miss on busRead missProcSharedConflict miss: place read miss on busRead missProcExclusiveConflict miss: write back block, place read miss on busWrite hitProcExclusiveWrite data in cacheWrite hitProcSharedPlace write miss on busWrite missProcInvalidPlace write miss on busWrite missProcSharedConflict miss: place write miss on busWrite missProcExclusiveConflict miss: write back, place write miss on busRead missBusSharedNo action; allow memory to respondRead missBusExclusivePlace block on bus; change to sharedWrite missBusSharedInvalidate blockWrite missBusExclusiveWrite back block; change to invalid 16 Title •Bullet