Sergey Legtchenko Xiaozhou Li Antony Rowstron Austin Donnelly Richard Black Storing Cold Data in the Cloud Cold data rarely accessed data Challenge storing cold data at low cost Pelican ID: 777283
Download The PPT/PDF document "Flamingo: Enabling Evolvable HDD-based N..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Flamingo: Enabling Evolvable HDD-based Near-Line Storage
Sergey Legtchenko, Xiaozhou Li, Antony Rowstron, Austin Donnelly, Richard Black
Slide2Storing Cold Data in the Cloud
Cold data: rarely accessed dataChallenge: storing cold data at low cost
Pelican
rack
(Microsoft,
[OSDI 14])
8% HDDs active
576 HDDs/server
Open ComputeCold Storage rack (Facebook)6% HDDs active240 HDDs/server
Custom racks:
trading latency for costOnly fraction of HDDs concurrently activeReduced #servers (1 or 2 per rack)
Benefits
Lower capital cost
Capped resource consumption
Higher storage density
Slide3Designing Cold Storage Racks is Hard
Resources are constrained in the rack
Experience from building Pelican
Design complexity
Storage stack is brittle to
design
changes
Impact
of resource provisioning on end performance?
Resource constraints:
1 HDD / cooling column
2 HDDs / tray
Vibration, bandwidth…
Power
Cooling
Software: co-designed, constraint-aware
Data Layout
IO Scheduler
Pelican: 8% HDDs active
Slide4Flamingo: a Tool to Help Cold Storage Rack Design
Input
Rack
Resource provisioning
specification
Rack description
Constraints
Hardware properties
Performance
goals
Set of rack descriptions:
same topology
varying resource provisioning
Resource provisioning
exploration
Perf.
Analysis
(simulator)
In the rest of the talk
Constraint Solver
Data layout,
IO scheduler
+
Data layout,
IO scheduler
+
Data layout,
IO scheduler
+
Storage stack configuration
Generic
Storage Stack
Data layout
IO scheduler
parameters
Online
Slide5{
D1,D2,D3,D4
}: 40
{D5,D6,D7,D8}: 40
{D4,D8}: 1
{D3,D7}: 1
{D1,D5}: 1
{D2,D6}: 1
Rack Description
Domain B:
type
:
vibration, budget: 1HDD
Domain A:
type
:
power,
budget
:
40W
Resource Domain:
set of HDDs sharing a limited resource
D1
D2
D4
D5
D6
D7
D8
D3
Standby
Active
Spin up
power: 10W
vibration: 1
power: 20W
vibration: 1
power: 2W
vibration: 0
HDD:
operating states + resource consumption
IO-capable state
40W
2 +
2 +
2 +
20 = 26
can be hard or soft
40W
2 +
2 +
20+
20 = 44W
Expresses constraints:
e.g.
o
nly 1 HDD can be spinning up in A
Slide6Storage Stack Configuration
Rack description
Constraint
Solver
Data layout
Groups of HDDs that concurrently transition state
Minimize inter-group conflicts
Slide7{G2,G1}: 1
Storage Stack Configuration
Rack description
Constraint
Solver
Data
layout
Groups of HDDs that concurrently transition state
Minimize inter-group conflicts
vibration, budget: 1
power
budget
:
40
D1
D2
D4
D5
D6
D7
D8
D3
D4
D8
Conflict
C
onflicts between groups
{D4,D8}: 1
{D3,D7}: 1
{D2,D6}: 1
{D1,D5}: 1
{D1,D2,D3,D4}: 40
{D5,D6,D7,D8}: 40
Rack description
{G1,G2}: 1
{G4,G3}: 1
{G3,G4}: 1
{G3,G4,G1,G2}: 40
{G4,G3,G2,G1}: 40
Inter-group
constraints
{G1,G2,G3,G4}: 40
{G1,G2}: 1
{G3,G4}: 1
Generic
Storage Stack
Data Layout
Spin ups/downs, IOs
Blob-store API
IO Scheduler
Blob Group of HDDs
D1
D2
D4
D5
D6
D7
D8
D3
D8
D3
vibration, budget: 1
power
budget
:
40
D7
D2
D6
D1
D5
D4
Each group conflicts with 2
D1
D2
D4
D5
D6
D7
D8
D3
D4
D8
Conflict
D1
D2
D4
D5
D6
D7
D8
D3
D8
D3
D7
D4
D6
D1
D5
D2
Conflicts minimized
vibration, budget: 1
power
budget
:
40
G1: {D3, D8}
G2: {D4, D7}
G3: {D1, D6}
G4: {D2, D5}
Group
definition
Sd
A
Su
1
0,
1
2
0,
1
2,
0
Slide8Resource Provisioning E
xplorationN resource types: N-dimensional space of rack descriptions
Resource 1
(e.g. power)
Resource 2 (e.g. vibration)
Fully provisioned
0
0
Fully provisioned
D1
D2
D4
D5
D6
D7
D8
D3
D4
D8
D1
D2
D4
D5
D6
D8
D3
40
1
40
1
D7
Slide9Resource Provisioning E
xplorationN resource types: N-dimensional space of rack descriptions
Resource 1
(e.g. power)
Resource 2 (e.g. vibration)
Fully provisioned
0
0
Fully provisioned
Fully-provisioned rack
(JBOD), per domain:all HDDs in most resource-consuming stateLeast-provisioned rack, per domain:
1 HDD in IO-capable state
n-1 in lowest resource-consuming
D1
D2
D4
D5
D6
D7
D8
D3
D4
D8
D1
D2
D4
D5
D6
D7
D8
D3
vibration, budget: 1
power
budget
:
26
Sd
A
Su
1
0,
1
2
0,
1
2,
0
D1
D2
D4
D5
D6
D7
D8
D3
D4
D8
D1
D2
D4
D5
D6
D7
D8
D3
vibration, budget: 2
power
budget
:
80
Relaxing
Bottleneck resource constraint
Slide10Resource Provisioning E
xplorationN resource types: N-dimensional space of rack descriptions
Resource 1
(e.g. power)
Resource 2 (e.g. vibration)
Fully provisioned
0
0
Fully provisioned
Fully-provisioned rack
(JBOD), per domain:all HDDs in most resource-consuming state
Bottleneck resource: vibration
Bottleneck resource: power
Discrete surface
in the N-dimensional space
For Pelican: 747 rack descriptions
Slide11Evaluation - Pelican
Pelican Simulator – Poisson workload, 1GB reads.
Pelican
Slide12Execution Time - Pelican
9
minutes
Slide13Execution Time - Pelican
9
minutes
Rack
name
OCP
Pelican
Rack_A
Rack_B
Rack_C
Rack_DRack_E#rack descriptions192174714211152973649683
Slide14Execution Time for Different Racks
3
hours
9
minutes
Rack
name
OCP
Pelican
Rack_A
Rack_BRack_CRack_DRack_E#rack descriptions1921
747
1421
1152
973
649
683
Slide15Conclusion
Cold storage racks: Co-design: resource-constrained hardware + constraint-aware softwarelow cost but
hard to (re)design
Flamingo simplifies design of
c
old storage racks
Synthesizes Data Layout and IO Scheduler parameters
Explores impact of resource provisioning on end performance
Redesign in days vs months manually