Large Scale Sky Computing Applications with Nimbus - PowerPoint Presentation

Slide1

Large Scale Sky Computing Applications with Nimbus

Pierre RiteauUniversité de Rennes 1, IRISAINRIA Rennes – Bretagne AtlantiqueRennes, FrancePierre.Riteau@irisa.fr

Slide2

Introduction ToSky

Computing

Slide3

IaaS clouds

On demand/elastic modelPay as you goAccess to virtual machines with administrator privilegesPortable execution stack

Commercial providersAmazon EC2 => “infinite”

resource pool (e.g. 10K cores)

Scientific clouds => limited number of resources

Science Clouds

FutureGrid

Nimbus clouds

Slide4

Sky Computing

Federation of multiple IaaS cloudsCreates large scale infrastructuresAllows to run software requiring large computational power

Slide5

Sky Computing

BenefitsSingle networking contextAll-to-all connectivitySingle security contextTrust between all entitiesEquivalent to local clusterCompatible with legacy code

Slide6

Large-Scale Sky

Computing Experiments

Slide7

Sky Computing

ToolkitNimbusResource managementContextualization (Context Broker)ViNeAll-to-all

connectivityHadoop

Task distributionFault

tolerance

Resource

dynamicity

Slide8

Context Broker

Service to configure a complete cluster with different rolesSupports clusters distributed on multiple clouds (e.g. Nimbus and Amazon EC2)VMs contact the context broker toLearn their roleLearn about other VMs in the clusterEx. : Hadoop master +

Hadoop slavesHadoop slaves configured to contact the master

Hadoop master configured to know the slaves

Slide9

Cluster description

<?xml version="1.0" encoding="UTF-8"?><cluster xmlns="http://www.globus.org/2008/06/workspace/metadata/logistics"> <workspace

> <

name>

hadoop-master

</

name

>

<image>fc8-i386-nimbus-blast-cluster-004</image>

<

quantity

>1</

quantity

> <nic

wantlogin="true">public</nic>

<ctx> <provides> <

identity /> <role

>hadoop_master</role

> <role>hadoop_slave

</role> </provides

> <requires> <identity

/>

<

role

name

="

hadoop_slave

"

hostname

="

true" pubkey="true" /> <role name="hadoop_master" hostname="true" pubkey="true" /> </requires> </ctx> </workspace>

<

workspace

>

<

name

>

hadoop-slaves

</

name

>

<image>fc8-i386-nimbus-blast-cluster-004</image>

<

quantity

>16</

quantity

>

<

nic

wantlogin

="

true

">public</

nic

>

<

ctx

>

<

provides

>

<

identity

/>

<

role

>

hadoop_slave

</

role

>

</

provides

>

<

requires

>

<

identity

/>

<

role

name

="

hadoop_master

"

hostname

="

true

"

pubkey

="

true

" />

</

requires

>

</

ctx

>

</

workspace

>

</cluster>

Slide10

ViNe

Project of the University of Florida (M. Tsugawa et al.)High performance virtual networkAll-to-all connectivity

ViNe

router

ViNe

router

ViNe

router

Slide11

Hadoop

Open-source MapReduce implementationHeavy industrial use (Yahoo, Facebook…)Efficient framework

for distribution of tasksBuilt-in

fault-toleranceDistributed file system (HDFS)

Slide12

Sky Computing Architecture

IaaS

Software

IaaS

Software

ViNe

Distributed

Application

Hadoop

MapReduce

App

Slide13

Grid’5000 Overview

Distributed over 9 sites in France~1500 nodes, ~5500 CPUsStudy of large scale parallel/distributed

systemsFeatures

Highly reconfigurableEnvironment

deployment

over

bare

hardware

Can

deploy

many

different

Linux distributionsEven other OS such

as FreeBSDControlableMonitorable (metrics

access)Experiments on all layersnetwork, OS, middleware, applications

Slide14

Grid’5000 Node Distribution

Slide15

FutureGrid: a Grid

TestbedNSF-funded experimental testbed~5000 cores6 sites connected

by a private network

Slide16

Resources used in

Sky Computing Experiments3 FutureGrid sites (US) with Nimbus installations

UCSD (San Diego)UF (Florida)UC (Chicago)Grid’5000 sites (France)

Lille (contains a white-listed

gateway

to

FutureGrid

)

Rennes, Sophia, Nancy, etc.

Grid’5000

is

fully

isolated from the Internet

One machine white-listed to access FutureGridViNe queue VR (Virtual Router) for

other sites

Slide17

ViNe

Deployment TopologySD

UF

UC

Lille

Rennes

Sophia

All-to-all

connectivity

!

White-listed

Queue VR

Grid’5000 firewall

Slide18

Experiment scenario

Hadoop sky computing virtual cluster already running in FutureGrid (SD, UF, UC)Launch BLAST MapReduce jobStart VMs on Grid’5000 resourcesWith contextualization to join the existing clusterAutomatically extend the

Hadoop clusterNumber of nodes increasesTaskTracker

nodes (Map/Reduce tasks execution)DataNode nodes (HDFS storage)

Hadoop

starts distributing tasks in Grid’5000

Job completes faster!

Slide19

Job progress with

cluster extension

Slide20

Scalable Virtual Cluster

Creation (1/3)Standard Nimbus propagation: scp

NimbusRepository

scp

scp

VMM A

VMM B

VM1

VM2

VM3

VM4

scp

scp

Slide21

Scalable Virtual Cluster Creation

(2/3)Pipelined Nimbus propagation: Kastafior/TakTuk

Nimbus

Repository

VMM A

VMM B

VM1

VM2

VM3

VM4

Propagate

Propagate

Propagate

Propagate

Slide22

Scalable Virtual Cluster Creation

(3/3)Leverage Xen Copy-on-Write (CoW

) capabilities

VMM A

Local cache

CoW

Image

4

Backing

file

VMM B

CoW

Image

3

Backing

file

Local cache

CoW

Image

2

Backing

file

CoW

Image

1

Backing

file

Slide23

Slide24

Conclusion

Slide25

Conclusion

Sky Computing to create large scale distributed infrastructuresOur approach relies onNimbus for

resource management, contextualization and

fast cluster instantiation

ViNe

for

all-to-all

connectivity

Hadoop

for

dynamic

cluster extension

Provides

both infrastructure and application elasticity

Slide26

Ongoing & Future Works

Elastic MapReduce implementation leveraging Sky Computing infrastructures (presented at

CCA ‘11)Migration support in Nimbus

Leverage spot instances in Nimbus

Slide27

Acknowledgments

Tim Freeman, John Bresnahan, Kate Keahey, David LaBissoniere (Argonne/University of Chicago)

Maurício Tsugawa, Andréa Matsunaga, José Fortes (University of Florida)Thierry

Priol, Christine Morin (INRIA)

Slide28

Thank you

!Questions?