Windows - PowerPoint Presentation

360 views
Uploaded On 2015-10-28

Windows - PPT Presentation

Azure Storage A Highly Available Cloud Storage Service with Strong Consistency Brad Calder Ju Wang Aaron Ogus Niranjan Nilakantan Arild Skjolsvold Sam McKelvie Yikang Xu Shashwat Srivastav ID: 175173

layer partition storage extent partition layer extent storage data stream index secondary server block stamp load primary append windows

Link:

Copy

Embed:

<iframe width="560" height="315" src="https://www.docslides.com/embed/175173" frameborder="0" allowfullscreen></iframe>

Download Presentation from below link

Download Presentation The PPT/PDF document "Windows" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

Slide1

Windows Azure Storage – A Highly Available Cloud Storage Service with Strong Consistency

Brad Calder,

Wang, Aaron Ogus, Niranjan Nilakantan, Arild Skjolsvold, Sam McKelvie, Yikang Xu, Shashwat Srivastav,

Jiesheng

Wu, Huseyin Simitci,

Jaidev

Haridas, Chakravarthy Uddaraju, Hemal Khatri, Andrew Edwards, Vaman Bedekar, Shane Mainali, Rafay Abbasi, Arpit Agarwal,

Mian

Fahim ul Haq, Muhammad

Ikram

ul Haq, Deepali Bhardwaj, Sowmya Dayanand, Anitha Adusumilli, Marvin McNett, Sriram Sankaran, Kavitha Manivannan, Leonidas Rigas

Microsoft CorporationSlide2

Windows Azure Storage – AgendaWhat it is and Data AbstractionsArchitecture and How it Works

Storage Stamp

Partition Layer

Stream Layer

Design Choices and Lessons LearnedSlide3

Windows Azure Storage

Geographically Distributed across 3 Regions

Anywhere at Anytime Access to data

>200

Petabytes

of raw storage by December 2011Slide4

Windows Azure Storage Data Abstractions

Blobs

– File system in the cloud

Tables

– Massively scalable structured storageQueues – Reliable storage and delivery of messages

Drives – Durable NTFS volumes for Windows Azure applicationsSlide5

Windows Azure Storage High Level ArchitectureSlide6

Design Goals

Highly Available with Strong Consistency

Provide access to data in face of failures/partitioning

Durability

Replicate data several times within and across data centers

ScalabilityNeed to scale to exabytes and beyondProvide a global namespace to access data around the worldAutomatically load balance data to meet peak traffic demandsSlide7

Windows Azure Storage Stamps

Storage Stamp

Storage

Location

Service

Access blob storage via the URL: http://<account>.blob.core.windows.net/

Data access

Partition Layer

Front-Ends

Stream Layer

Intra-stamp replication

Storage Stamp

Partition Layer

Front-Ends

Stream Layer

Intra-stamp replication

Inter-stamp (Geo) replicationSlide8

Storage Stamp Architecture – Stream Layer

Append-only distributed file system

All data from the Partition Layer is stored into files (extents) in the Stream layer

An extent is replicated 3 times across different fault and upgrade domains

With random selection for where to place replicas for fast MTTR

Checksum all stored dataVerified on every client read

Scrubbed every few daysRe-replicate on disk/node/rack failure or checksum mismatch

Extent Nodes (EN)

Paxos

Stream

Layer

(Distributed

File System)Slide9

Storage Stamp Architecture – Partition Layer

Provide transaction semantics and strong consistency for Blobs, Tables and Queues

Stores and reads the objects to/from extents in the Stream layer

Provides inter-stamp (geo) replication by shipping logs to other stamps

Scalable object index via partitioning

Extent Nodes (EN)

Paxos

Partition

Server

Partition

Server

Partition

Server

Partition

Server

Partition

Master

Lock Service

Partition Layer

Stream

LayerSlide10

Storage Stamp Architecture

Stateless Servers

Authentication + authorization

Request routing

Extent Nodes (EN)

Paxos

Front End Layer

Partition

Server

Partition

Server

Partition

Server

Partition

Server

Partition

Master

Lock Service

Partition Layer

Stream

LayerSlide11

Storage Stamp Architecture

Extent Nodes (EN)

Paxos

Front End Layer

Incoming Write Request

Partition

Server

Partition

Server

Partition

Server

Partition

Server

Partition

Master

Lock Service

Ack

Partition Layer

Stream

LayerSlide12

Partition LayerSlide13

Partition Layer – Scalable Object Index

100s of Billions of

blobs,

entities, messages

across all accounts

can be stored in a single stamp Need to efficiently enumerate, query, get, and update them

Traffic pattern can be highly dynamicHot objects, peak load, traffic bursts, etcNeed a scalable index for the objects that can

Spread the index across 100s of serversDynamically load balanceDynamically change what servers are serving each part of the index based on loadSlide14

Scalable Object Index via Partitioning

Partition Layer maintains an internal Object Index Table for each data abstraction

Blob Index: contains all blob objects for all accounts in a stamp

Table Entity Index: contains all entities for all accounts in a stamp

Queue Message Index: contains all messages for all accounts in a stamp

Scalability is provided for each Object IndexMonitor load to each part of the index to determine hot spots

Index is dynamically split into thousands of Index RangePartitions based on loadIndex RangePartitions are automatically load balanced across servers to quickly adapt to changes in loadSlide15

Account

Name

Container

Name

Blob

Name

aaaa

aaaaa

……..

zzzz

zzzzz

Split index into RangePartitions based on load

Split at

PartitionKey

boundaries

PartitionMap

tracks Index RangePartition assignment to partition servers

Front-End caches the

PartitionMap

to route user requests

Each part of the index is assigned to only one Partition Server at a time

Storage Stamp

Partition

Server

Partition

Server

Account

Name

Container

Name

Blob

Name

richard

videos

tennis

………

zzzz

zzzzz

Account

Name

Container

Name

Blob

Name

harry

pictures

sunset

………

richard

videos

soccer

Partition

Server

Partition Master

Partition Layer – Index Range Partitioning

Front-End

Server

PS 2

PS 3

PS 1

A-H:

PS1

H’-R: PS2

R’-

PS3

A-H:

PS1

H’-R: PS2

R’-

PS3

Partition

Map

Blob Index

Partition Map

Account

Name

Container

Name

Blob

Name

aaaa

aaaaa

………

harry

pictures

sunrise

A-H

R’-Z

H’-RSlide16

Each RangePartition – Log Structured Merge-Tree

Checkpoint

File Table

Checkpoint

File Table

Checkpoint

File Table

Row Data Stream

Blob Data

Blob Data Stream

Commit Log Stream

Metadata log Stream

Persistent Data (Stream Layer)

Row Cache

Index Cache

Bloom Filters

Load Metrics

Memory Table

Memory Data

Writes

Read/QuerySlide17

Stream LayerSlide18

Stream LayerAppend-Only Distributed File SystemStreams are very large files

Has file system like directory namespace

Stream Operations

Open, Close, Delete Streams

Rename Streams

Concatenate Streams togetherAppend for writingRandom readsSlide19

Extent E2

Extent E3

Block

Stream Layer Concepts

Block

Min unit of write/read

Checksum

Up to N bytes (e.g. 4MB)

Extent

Unit of replication

Sequence of blocks

Size limit (e.g. 1GB)

Sealed/unsealed

Stream

Hierarchical namespace

Ordered list of pointers to extents

Append/Concatenate

Block

Extent E4

Stream //

foo

myfile.data

Ptr E1

Ptr E2

Ptr E3

Ptr E4

ealed

unsealed

ealed

unsealed

ealed

unsealed

Extent E1Slide20

Creating an Extent

Stream Master

Paxos

Partition Layer

EN 1

EN 2

EN 3

Create Stream/Extent

Allocate Extent replica set

Primary

Secondary A

Secondary B

EN1 Primary

EN2, EN3 SecondarySlide21

Replication Flow

Paxos

Partition Layer

EN 1

EN 2

EN 3

Append

Primary

Secondary A

Secondary B

Ack

EN1 Primary

EN2, EN3 SecondarySlide22

Providing Bit-wise Identical ReplicasWant all replicas for an extent to be bit-wise the same, up to a committed length

Want to store pointers from the partition layer index to an

extent+offset

Want to be able to read from any replica

Replication flow

All appends to an extent go to the PrimaryPrimary orders all incoming appends and picks the offset for the append in the extentPrimary then forwards offset and data to secondariesPrimary performs in-order acks back to clients for extent appendsPrimary returns the offset of the append in the extent

An extent offset can commit back to the client once all replicas have written that offset and all prior offsets have also already been completely writtenThis represents the committed length of the extentSlide23

Dealing with Write Failures

Failure during append

Ack

from primary lost when going back to partition layer

Retry from partition layer can cause multiple blocks to be appended (duplicate records)

Unresponsive/Unreachable Extent Node (EN)

Append will not be

acked

back to partition layer

Seal the failed extent

Allocate a new extent and append immediately

Stream //

foo

/myfile.dat

Ptr E1

Ptr E2

Ptr E3

Ptr E4

Extent E5

Ptr E5

Extent E1

Extent E2

Extent E3

Extent E4Slide24

Extent Sealing (Scenario 1)

Stream Master

Paxos

Partition Layer

EN 1

EN 2

EN 3

EN 4

Append

Primary

Secondary A

Secondary B

Ask for current length

120

Sealed at 120

Seal Extent

Seal ExtentSlide25

Extent Sealing (Scenario 1)

Stream Master

Paxos

Partition Layer

EN 1

EN 2

EN 3

EN 4

rimary

Secondary A

Secondary B

Sync with SM

120

Sealed at 120

Seal ExtentSlide26

Extent Sealing (Scenario 2)

Paxos

Partition Layer

EN 1

EN 2

EN 3

EN 4

Append

Primary

Secondary A

Secondary B

Ask for current length

120

Sealed at 100

Seal Extent

100

Seal ExtentSlide27

Extent Sealing (Scenario 2)

Paxos

Partition Layer

EN 1

EN 2

EN 3

EN 4

Primary

Secondary A

Secondary B

Sync with SM

Sealed at 100

Seal Extent

100Slide28

Providing Consistency for Data Streams

EN 1

EN 2

EN 3

Primary

Secondary A

Secondary B

Partition Server

Network partition

PS can talk to EN3

SM cannot talk to EN3

For Data Streams, Partition Layer only reads from offsets returned from successful appends

Committed on all replicas

Row and Blob Data Streams

Offset valid on any replica

Safe to read from EN3Slide29

Providing Consistency for Log Streams

EN 1

EN 2

EN 3

Primary

Secondary A

Secondary B

Partition Server

Check commit length

Logs are used on partition load

Commit and Metadata log streams

Check commit length first

Only read from

Unsealed replica if all replicas have the same commit length

A sealed replica

Check commit length

Seal Extent

Use EN1, EN2 for loading

Network partition

PS can talk to EN3

SM cannot talk to EN3Slide30

Our Approach to the CAP Theorem

Layering and co-design provides

extra

flexibility to achieve “C” and “A” at same time while being partition/failure tolerant for our fault model

Stream Layer

Availability with Partition/failure toleranceFor Consistency, replicas are bit-wise identical up to the commit lengthPartition LayerConsistency with Partition/failure toleranceFor Availability, RangePartitions can be served by any partition server and are moved to available servers if a partition server failsDesigned for specific

classes of partitioning/failures seen in practiceProcess to Disk to Node to Rack failures/unresponsivenessNode to Rack level network partitioningSlide31

Design Choices and Lessons LearnedSlide32

Design ChoicesMulti-Data

rchitecture

Use extra resources to serve mixed workload for incremental costs

Blob -> storage capacity

Table -> IOpsQueue -> memory

Drives -> storage capacity and IOpsMultiple data abstractions from a single stackImprovements at lower layers help all data abstractionsSimplifies hardware management

Tradeoff: single stack is not optimized for specific workload patternAppend-only SystemGreatly simplifies replication protocol and failure handlingConsistent and identical replicas up to the extent’s commit length

Keep snapshots at no extra costBenefit for diagnosis and repairErasure CodingTradeoff: GC overhead

Scaling Compute Separate from StorageAllows each to be scaled separately

Important for multitenant environmentMoving toward full bisection bandwidth between compute and storageTradeoff: Latency/BW to/from storageSlide33

Lessons LearnedAutomatic load balancingQuickly adapt to various traffic conditions

Need to handle every type of workload thrown at the system

Built an easily tunable and extensible language to dynamically tune the load balancing rules

Need to tune based on many dimensions

CPU, Network, Memory,

tps, GC load, Geo-Rep load, Size of partitions, etcAchieving consistently low append latenciesEnded up using journalingEfficient upgrade supportPressure point testingSlide34

Windows Azure Storage Summary

Highly Available Cloud Storage with Strong Consistency

Scalable data abstractions to build your applications

Blobs – Files and large objects

Tables – Massively scalable structured storage

Queues – Reliable delivery of messages

Drives – Durable NTFS volume for Windows Azure applications

More information

Windows Azure tutorial this Wednesday 26

, 17:00 at start of SOCC

http://blogs.msdn.com/windowsazurestorage/