Íñigo Goiri William Katsak Md E Haque Kien Le Ryan Beauchea Jordi Guitart Jordi Torres Thu D Nguyen Ricardo Bianchini Department of Computer Science Motivation Datacenters consume large amounts of energy ID: 816434
Download The PPT/PDF document "GreenSoftware : Managing Datacenters Pow..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
GreenSoftware:Managing Datacenters Powered by Renewable Energy
Íñigo Goiri, William Katsak, Md E Haque, Kien Le,Ryan Beauchea, Jordi Guitart, Jordi Torres,Thu D. Nguyen, Ricardo BianchiniDepartment of Computer Science
Slide2MotivationDatacenters consume large amounts of energyHigh energy cost and carbon footprintBrown electricity: coal and natural gasConnect datacenters to green sources: solar, wind
Apple DC in Maiden, NC
4
0MW solar farm
2
Green datacenter
Slide3Challenges and opportunitiesScheduling workload/energy sourcesLower costs: brown energy, peak brown power, capitalStudy opportunities in green datacentersBuild hardware/
softwarePower
Time
Load
Variable
3
Solar power
Workload
Slide4GreenSoftwareHow to build software for green datacenters?Malleable energy demand
Idle nodes → Turn off/Sleep (S3) [COLP’01]Reduce frequency (DVFS) → Lower qualityMove computation under renewablesWeather forecast → Green energy forecastDelay computation or degrade qualityLeverage energy storage4
Slide5OutlineMotivationGreenSoftwareGreenSlotGreenHadoop
GreenSwitchGreenCassandra… and othersConclusion5
Slide6GreenSlot [SC’11]Batch jobs on SLURM (& Hadoop)Send idle nodes to
S3Predict solar availabilityDelay jobs within deadlinesKnown jobs characteristics (length, deadline, size…)Heuristic6
Time
Job 1
Power
Deadline
Job 2
Job 3
Job 4
Slide7GreenSlot [SC’11]Batch jobs on SLURM (& Hadoop)Send idle nodes to
S3Predict solar availabilityDelay jobs within deadlinesKnown jobs characteristics (length, deadline, size…)Heuristic7
Time
Job 1
Power
Deadline
Job 2
Job 3
Job 4
Slide8GreenHadoop [Eurosys’12]Batch jobs on HadoopSend idle nodes to S3
Make required data availableMove data blocksPredict solar availabilityDelay jobs within deadlinesPredict global jobs energy consumptionHeuristic8
Map
1
Map
2
Map
3
Map
4
Map
5
Reduce
Reduce
6
7
Shuffle
Slide9Covering subset
GreenHadoop
: Data management
Deactivate servers to save energy
Some data might become unavailable
Prior solution: covering subset [Leverich’09]
Set of servers always running has
ALL
data
9
7
3
4
5
2
1
6
8
7
1
4
56328173Our approachOnly required data has to be availableWe usually require fewer active serversServerBlock
Slide10GreenHadoop: Data managementServer 1
172
Active
Decommission
Down
Server 2
4
3
5
6
Server 3
4
6
Required file
Non-required file
Server 4
2
3
8
4
Server 5
367JobA4JobB5JobC16Running queue:10
Slide11GreenHadoop: Data managementServer 4
238
4
Server 5
3
6
7
Active
Decommission
Down
GreenHadoop
(computation) requires only 2 servers
Server 1
1
7
2
Server 1
1
7
2
Server 2
4356Server 346Required fileNon-required fileJobA4JobB5JobC16Running queue:11
Slide12GreenHadoop: Data managementActive
DecommissionDownMove required files to Active serversServer 1
1
7
2
Server 2
4
3
5
6
Server 3
4
6
1
Server 4
2
3
8
4
Server 5
3
67ReplicateJobA4JobB5JobC16Running queue:12
Slide13Server 1
172GreenHadoop: Data management
Active
Decommission
Down
Decommissioned server can be sent to Down
Server 1
1
7
2
Server 2
4
3
5
6
Server 3
4
6
Required file
Non-required file
1
Server 42384Server 5367JobA4JobB5JobC16Running queue:13
Slide14Server 1
172GreenHadoop: Data management
Active
Decommission
Down
Jobs to be executed change → Required files change
Server 2
4
3
5
6
Server 3
4
6
Non-required file
1
Server 4
2
3
8
4
Server 5367JobA4JobB5JobC16JobD8Required file6464648Required fileRunning queue:14
Slide15Server 4
238
4
Server 1
1
7
2
GreenHadoop
: Data management
Active
Decommission
Down
Make missing data available
Server 2
4
3
5
6
Server 4
2
3
84Server 5367Server 3461Required fileNon-required fileJobB5JobC1JobD8Required fileRunning queue:15
Slide16Server 4
238
4
Server 1
1
7
2
GreenHadoop
: Data management
Active
Decommission
Down
Server 2
4
3
5
6
Server 4
2
3
8
4Server 5367GreenHadoop (computation) requires 3 serversServer 3461Non-required fileJobB5JobC1JobD8Required fileRunning queue:16
Slide17GreenSwitch [ASPLOS’13]Batch jobs on HadoopSimilar to
GreenHadoopEnergy storageBattery Net meteringSchedule workload and energy sourcesOptimizationEvaluation on Parasol(Presented on Monday by Thu)17
Slide18GreenCassandraDistributed DB/storage on CassandraAdd an optional ring
Degrade quality when no green181
4
6
2
3
5
DHT Ring
A
A
A
1
4
3
5
6
2
Double
DHT Ring
Optional
A
AServerData
Slide19GreenSoftware summaryType
Malleable energyGreen adaptabilityGreenSlotBatch jobsDelay jobsSleep serversDelay until greenGreenHadoopBatch jobs
Delay jobs
Sleep servers
Data management
Delay until
green
GreenSwitch
Batch/interactive jobs
Delay jobsSleep servers
Delay until
green
Energy storage
GreenCassandra
Distributed
storage
Optional ring
Degrade quality
GreenSLA
VMs
Migrate VMs
Sleep serversRoute green energyto racksGreenParMPI jobsChange parallelismSleep serversGreater parallelismon greenGreenScaleNon-deferrable jobsCPU and mem DVFSFaster on greenGreenNebulaGeo distributed VMsMigrate VMs“Follow the renewables”19
Slide20ConclusionsGreen datacentersChallenges & opportunitiesHardware/software solutionGreenSoftware
Adapt software to green datacentersMalleable energy demandMatch computation and renewables20
Slide21GreenSoftware:Managing Datacenters Powered by Renewable Energy
Íñigo Goiri, William Katsak, Md E Haque,
Kien
Le,
Ryan Beauchea, Jordi Guitart, Jordi Torres,
Thu D. Nguyen, Ricardo Bianchini
Department of Computer Science
Slide22Other GreenSoftwareGreenSLA [IGCC’13]Bringing green energy to usersNew hardware to route green energyGreenPar
MPI jobs with sub linear speedupUse “Free” green energyGreenNebulaVMs in multiple geo distributed datacentersFollow the sunGreenScaleChange frequency (DVFS)22
Slide23GreenParMPI jobs on VMsAdd more computation capacityLower energy-efficiencyUse available “green”
23Computation capacitySpeedup
Power
Time
Slide24GreenSLA[IGCC’13]HPC jobs on VMsUsers require % of green energyHardware to bring green energy to
VMsSchedulingAssign green energy to racksMove VMs among racksHeuristic24
Slide25Default Green DatacenterSolar
InverterPDU
Rack3
PDU
Rack2
PDU
Rack1
Mixed
Bus
Datacenter Power
Distribution
Slide26GreenSLA
:
Proposed Power Distribution
PDU
Rack3
PDU
Rack2
PDU
Rack1
Battery
Mixed
Bus
Control Module
Charge Controller
Green
Bus
S
S
S
Solar
InverterDatacenter Power Distribution
Slide27Parasol without GreenSwitch
Green useGreen availableNet meteringBrown useIT load
27
Slide28GreenSwitch: deferrable workload
Battery dischargeBattery chargeIT load
Net metering
28
Green available
Green use