/
1 Parallel and Multiprocessor Architectures – Shared Memory 1 Parallel and Multiprocessor Architectures – Shared Memory

1 Parallel and Multiprocessor Architectures – Shared Memory - PowerPoint Presentation

cheryl-pisano
cheryl-pisano . @cheryl-pisano
Follow
353 views
Uploaded On 2018-11-09

1 Parallel and Multiprocessor Architectures – Shared Memory - PPT Presentation

Recall Microprocessors are classified by how memory is organized Tightlycoupled multiprocessor systems use the same memory They are also referred to as shared memory multiprocessors The processors do not necessarily have to share the same block of physical memory ID: 724735

computing memory systems distributed memory computing distributed systems processor system multiprocessor architectures shared cache processors parallel update data cached

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "1 Parallel and Multiprocessor Architectu..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

1

Parallel and Multiprocessor Architectures – Shared Memory

Recall: Microprocessors are classified by how memory is organizedTightly-coupled multiprocessor systems use the same memory. They are also referred to as shared memory multiprocessors.

The processors do not necessarily have to share the same block of physical memory:

Each processor can have its own memory, but it must share it with the other processors.Configurations such as these are called distributed shared memory multiprocessors.Slide2

2

Parallel and Multiprocessor Architectures – Shared Memory

Also, each processor can have its local cache memory used with a single global memorySlide3

3

Parallel and Multiprocessor Architectures – Shared Memory

A type of distributed shared memory system is called a Shared Virtual Memory System

Each processor contains a cache

The system has no primary memoryData is accessed through cache directories maintained in each processorProcessors are connect via two unidirectional ringsThe level-one ring can connect 8 to 32 processorsThe level-two ring can connect up to 34 level-one ringsExample: If Processor A referenced data in location X, Processor B will place Address X’s data on the ring with a destination address of Processor A

Processor A

Processor B

Data at Address XSlide4

4

Parallel and Multiprocessor Architectures – Shared Memory

Shared memory MIMD machines can ALSO be divided into two categories based upon how they access memory or synchronize their memory operations.Uniform access approach

Non uniform access approachIn Uniform Memory Access (UMA) systems, all memory accesses take the same amount of time.

A switched interconnection UMA system becomes very expensive as the number of processors growBus-based UMA systems saturate when the bandwidth of the bus becomes insufficientMultistage UMA run into wiring constraints and latency issues as the number of processors growSymmetric multiprocessor UMA must be fast enough to support multiple concurrent accesses to memory, or it will slow down the whole system.The interconnection network of a UMA system limits the number of processors – scalability is limited.Slide5

5

Parallel and Multiprocessor Architectures – Shared Memory

The other category of MIMD machines are the Nonuniform Memory Access (NUMA) systems.

NUMA systems can overcome the inherent UMA problems by providing each processor with its own memory.Although the memory is distributed, NUMA processors see the memory as one contiguous addressable space.Thus, a processor can access its own memory much more quickly than it can access memory that is elsewhere.Not only does each processor have its own memory, it also has its own cache, a configuration that can lead to

cache coherence problems.Cache coherence problems arise when main memory data is changed and the cached image is not. (We say that the cached value is stale.)To combat this problem, some NUMA machines are equipped with snoopy cache controllers that monitor all caches on the systems. These systems are called cache coherent NUMA (CC-NUMA) architectures.A simpler approach is to ask the processor having the stale value to either void the stale cached value or to update it with the new value. Slide6

6

Parallel and Multiprocessor Architectures – Shared Memory

When a processor’s cached value is updated concurrently with the update to memory, we say that the system uses a write-through cache update protocol.

If the write-through with update protocol is used, a message containing the update is broadcast to all processors so that they may update their caches.

If the write-through with invalidate protocol is used, a broadcast asks all processors to invalidate the stale cached value. Write-invalidate uses less bandwidth because it uses the network only the first time the data is updated, but retrieval of the fresh data takes longer.Write-update creates more message traffic, but all caches are kept current. Another approach is the write-back protocol that delays an update to main memory until the modified cache block is replaced and written to memory.At replacement time, the processor writing the cached value must obtain exclusive rights to the data. When rights are granted, all other cached copies are invalidated.Slide7

7

Parallel and Multiprocessor Architectures – Distributed Computing

Distributed computing is another form of multiprocessing. However, the term distributed computing means different things to different people.In a sense, all multiprocessor systems are distributed systems because the processing load is distributed among processors that work collaboratively.

What is really meant by distributed system is, the processing units are very loosely-coupled. Each processor is independent with its own memory and cache, and the processors communicate via a high speed network.Another name for this is

Cluster Computing

Processing units connected via a bus are considered

tightly-coupled

.

Grid Computing

is an example of distributed computing – make use of heterogeneous CPUs and storage devices located in different domains to solve computation problems too large for any single supercomputer

The difference between Grid Computing and Cluster Computing is that Grid Computing can use resources in different domains versus only the same domain.

Global Computing

is grid computing with resources provided by volunteers.Slide8

8

Parallel and Multiprocessor Architectures – Distributed Systems

For general-use computing, transparency is important – details about the distributed nature of the system should be hidden.Using remote system resources should require no more effort than a local system.An example of this type of distributed system is called

Ubiquitous computing systems (or Pervasive computing systems).These systems are totally embedded in the environment – simple to use – completely connected – mobile – invisible and in the background.Remote procedure calls (RPCs) enable this transparency. RPCs use resources on remote machines by invoking procedures that reside and are executed on the remote machines.

RPCs are employed by numerous vendors of distributed computing architectures including the Common Object Request Broker Architecture (CORBA) and Java’s Remote Method Invocation (RMI).Slide9

9

Parallel and Multiprocessor Architectures – Distributed Systems

Cloud computing is distributed computing to the extreme.It provides services over the Internet through a collection of loosely-coupled systems.

In theory, the service consumer has no awareness of the hardware, or even its location.Your services and data may even be located on the same physical system as that of your business competitor.The hardware might even be located in another country.

Security concerns are a major inhibiting factor for cloud computing.