Aung Oo Lead Principal Program Manager Azure Storage Microsoft Corporation Sirius Kuttiyan Principal Program Manager Azure Storage Microsoft Corporation Objectives Features and capabilities provided by Premium Storage ID: 654984
Download Presentation The PPT/PDF document "Microsoft Azure Premium Storage" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1Slide2
Microsoft Azure Premium Storage
Aung OoLead Principal Program Manager, Azure StorageMicrosoft Corporation
Sirius Kuttiyan
Principal Program
Manager, Azure Storage
Microsoft CorporationSlide3
ObjectivesFeatures and capabilities provided by Premium Storage
Inner workings of Premium StorageReal world workloads and best practicesTakeawaysPremium Storage enables industry leading high IO performance for IaaS VMsRun demanding enterprise workloads including Databases, Big Data
and Data Warehousing
on Azure
Session Objectives
And
TakeawaysSlide4
Agenda
OverviewInside Premium StorageDemoBest Practices and Patterns
Q&ASlide5
OverviewSlide6
What is Azure Premium Storage?
Azure Storage provides persistent data disks for all IaaS VMsTwo types of durable storage: Standard Storage and Premium Storage
High I/O performance and low latency
SSD-based hardware
Ideal for
I/O intensive
workloadsSlide7
Key Features/Capabilities
High scale VMs>64,000 IOPS32 TB of premium storage dataUp to 512 MB/sec disk throughput
Highly durable
and available
3 replicas, strong consistentSlide8
Available now in 6 regions. More coming!
West Europe
East US 2
West US
Southeast Asia
West Japan
East ChinaSlide9
Virtual Machines for Premium StorageDS-Series VM
New VM class to support Premium Storage DiskBlob Cache - RAM and Local SSD used for cachingLocal SSD for application useMixed disk sizes on the same VM
VM Size
CPU Cores
RAM
Local Disk
Size
Max.
# Disks
Cache
Size
Max.
Disk
Performance
Max
. Cache
Performance
Standard_DS14
16
112 GB
224 GB
32
576 GB
50,000 IOPS512 MB/s64,000 IOPS
524 MB/sSlide10
Premium Storage DisksScalability and Performance
Provisioned Disk IOPS and Throughput
Storage Disk Type
P10
P20
P30
Disk size
128
GiB
512
GiB
1024
GiB
(1 TB)
IOPS per disk
500
2,300
5,000
Throughput per disk
100 MB per
second
150 MB per second
200 MB per secondSlide11
Pricing
Premium StorageProvisioned disk size (Pro-rated hourly)
Snapshots on Premium Storage
Storage Disk Type
P10
P20
P30
Disk size
128
GiB
512
GiB
1024
GiB
(1 TB)
Price*
$17.32
$66.56
$122.88
IOPS per disk
500
2,300
5,000
Throughput per disk
100 MB per second
150 MB per second200 MB per second
*Vary per regionSlide12
Inside Premium StorageSlide13
Premium Storage Design
DS-Series VM supports disk caching
Blobcache
, caching layer
Persistent data on our industry leading existing LRS with three replicas
Virtual Machine
RAM Cache
Server SSD Cache
Blobcache
Azure Storage Page BlobsSlide14
Cache Hit
Cache MissPremium Storage Design: Three-Level System
DS-Series VM
Uncached
Premium Storage Disk
Cached Premium Storage Disk
Local
Disk
Disk-Level Provisioning
Disk-Level Provisioning
SSD-Level Provisioning
VM Network-Level Provisioning
Azure Storage Page Blobs
Server SSDSlide15
DemoSlide16
Cache Hit
Premium Storage DemoScenario 1: 50K write IOPS with 10 uncached disksScenario 2: 64 K read IOPS with 1 cached disk Scenario 3:100K+ IOPS Combined Scenario 1 & 2
Azure Storage Page Blobs
Server SSD
5,000 IOPs, 200 MB/sec each disk
Uncached
disks
Uncached
disks
Uncached
disks
Uncached
disks
Uncached
disks
Uncached
disks
Uncached
disks
50,000 IOPs, 512 MB/sec – 10 disks
Cache Miss
DS14 VM
Uncached
disks
Uncached
disks
Uncached
disks
Scenario 1
Scenario 2
Scenario 3
64,000 IOPS, 524 MB/sec
Local
DiskSlide17
Best Practices and PatternsSlide18
Topics
Service Specifications
VM and Disk sizes and limits
IOPS and Throughput
Traffic Patterns
IO Size
Queue Depth
Cache hits
Migration
Cost Optimization
Migration tips
Backup/DR
Backing up the VHDs
Azure BackupSlide19
VM and Disk sizesFor Premium Storage
VM sizesStorage Disk TypeP10P20P30Disk size128
GiB
512 GiB
1024
GiB
(1 TB)
IOPS per disk
500
2300
5000
Throughput per disk
100 MB per
second
150 MB per
second
200 MB per
second
VM Size
CPU cores
Max.
Disk IOPS
Max. Disk Bandwidth
Cache IO limits
STANDARD_DS11
3,20032 MB per second4000 IOPS and
33 MB/sec per core(Combined limits for cache and local SSD)
STANDARD_DS22
6,40064 MB per second
STANDARD_DS3
412,800128 MB per second
STANDARD_DS4
825,600
256 MB per secondSTANDARD_DS11
26,400
64 MB per secondSTANDARD_DS12
412,800
128 MB per second
STANDARD_DS138
25,600256 MB per second
STANDARD_DS1416
50,000512 MB per second
Disk sizesSlide20
Understanding the VM limitExample #1: DS1 VM with a P30 (1 TB) data disk
Disk limit
VM limit
IOPS/Throughput
Limits
IOPS
Throughput
P30 Disk
5,000
200 MB/sec
DS1 VM
3,200
32 MB/sec
VM
limit
constrains
the
performanceSlide21
Understanding the Disk limitExample #2: DS4 VM with a P30 (1 TB) data disk
Disk limit
VM limit
IOPS / Throughput
Limits
IOPS
Throughput
P30 Disk
5,000
200 MB/sec
DS4 VM
25,600
256 MB/sec
Disk limit
constrains the performanceSlide22
VM Limit vs. Disk Limit… continuedExample #3: Pushing the limits
DS14 VM with 32 x P30 (1 TB) data disksLimitsIOPSThroughput
32 x P30 Disks (theoretical)
160,000
6,400 MB/sec
DS14 VM
50,000
512 MB/sec
VM
limit
constrains
the
performance.Slide23
IOPS and ThroughputExample: 1,000 IOPS with 8KB IO Size
Throughput = 1,000 IOPS * 8 KB = 8 MB/secIOPS Limit is separate from Throughput limitEnforced independentlyThrottling kicks in when either of the limits are reachedSlide24
Topics
Service Specifications
VM and Disk sizes and limits
IOPS and Throughput
Traffic Patterns
IO Size
Queue Depth
Cache hits
Migration
Cost Optimization
Migration tips
Backup/DR
Backing up the VHDs
Azure Backup
PSlide25
IO Size Implications
IO Size implicationsUse smaller IO sizes to increase IOPSUse larger IO sizes to increase ThroughputExample #1: P30 disk; IO Size is 8KB; 5000 IOPSThroughput = 5000 * 8 KB = 40 MB/sec
Example #2: P30 disk; IO Size is 256 KB
P30 disk Throughput limit is 200 MB/sec
Max IOPS = 200 / 0.25 = 800 IOPSSlide26
Queue Depth“QD” is the Number of pending IO requests
Example: If the latency is 1 millisecond, what is the QD needed to achieve 5000 IOPS?QD = 5000 * 0.001 = 5Slide27
Queue Depth OptimizationKeep IOPS & Throughput just below the limits
Control QD / IO Channels / ThreadsQD for Striped volumes = QD Per Disk x Number of Columns
If QD is too high:
If QD is too low:Slide28
Cache hitsCache hits don’t consume disk IOPS or Throughput
Cache has separate limits: 4000 IOPS per core; 33 MB/sec per coreShared by Cache as well as local SSD IOsAll IOs on cached disks consume Cache limits
Recommended disk cache settings
“
ReadWrite
” for OS disks by default
“ReadOnly” for data disks by default
“None” for write-only / write-heavy data disksSlide29
More IOPS and Throughput using Cache hitsExample #1: DS14 VM with a single P30 disk
5,000 IOPS and 200 MB/sec for Writes64,000 IOPS and 524 MB/sec for cache ReadsExample #2: Pushing beyond the limits…DS14 VM with 1 x P30 disk ReadOnly cache and 10
x P30
disks with No cache
100,000
IOPS and
1000 MB/secSlide30
SQL DemoSlide31
SQL PerformanceTest Results
OperationExecution TimeBandwidthFull Restore
49min
266MB/sec
Full Backup
52min
249 MB/sec
Index Scan select COUNT*
25min
470MB/sec
Drop Index (Clustered)
23min
525MB/sec
Create Index (Clustered)
1hr 39min
252 MB/sec
Checkdb
54min
269MB/sec
Configuration
DS14 VM (16 cores and 116 GB RAM). Ten P30 (1 TB) data disks.
Database with ~750GB data, which is larger than the cache on the VM (576 GB).
Single table (7.2 billon rows)/single clustered index (750GB)
Up to 140% faster than competitionSlide32
Topics
Service Specifications
VM and Disk sizes and limits
IOPS and Throughput
Traffic Patterns
IO Size
Queue Depth
Cache hits
Migration
Cost Optimization
Migration tips
Backup/DR
Backing up the VHDs
Azure Backup
P
PSlide33
Cost OptimizationThree specific disk size offers
Pricing includes GB as well as IOPS/ThroughputOptimizing the cost / configurationDetermine the GB, IOPS and Throughput neededSelect the disk offering required for meeting thatExample #1: 1 TB and 10,000 IOPS
Use 2 x P30 disks striped to create a single volume with 10,000 IOPS
Cost for competitive offer is over 30%
more than
Premium
StorageSlide34
Migration from Standard to PremiumSelecting the VM size
Choose based on VM specifications and application needsSmaller VMs may be sufficient due to better disk performanceExample #1: Migrating a D4 VM (8 cores; 16 disks)D4 VM has max disk IOPS of 16 * 500 = 8000 IOPSDS3 VM (4 cores) can provides 12,800 IOPS for disk traffic
Also consider CPU, Memory and Bandwidth needsSlide35
Migration to Premium… continuedRe-evaluate the disk sizes
Optimize for Premium Storage based on Capacity, IOPS and ThroughputConsider peak load and future growthDisk sizes can be increased laterExample #2: You have 4 x Standard 1 TB disksStriped to get a volume with up to 2000 IOPSYou might be able to replace it with a single P20 or P30 disk depending on capacity requirements
Choose the right cache settings per diskSlide36
VM / Disk Migration stepsFor details, refer to Migration guidelines dochttp
://azure.microsoft.com/en-us/documentation/articles/storage-migration-to-premium-storage/ Also look at application migration documentsNote a few points…DS-series VM cannot be placed in an existing Resource Group / Cloud ServiceFor reusing an existing Virtual Network, it needs to be a Regional Virtual Network
35 TB limit on provisioned capacity per storage account
For custom VM image deployment, max size allowed is 17 TBSlide37
Linux Support
DistributionVersionSupported KernelSupported Image
Ubuntu
12.04
3.2.0-75.110
Ubuntu-12_04_5-LTS-amd64-server-20150119-en-us-30GB
14.04
3.13.0-44.73
Ubuntu-14_04_1-LTS-amd64-server-20150123-en-us-30GB
14.10
3.16.0-29.39
Ubuntu-14_10-amd64-server-20150202-en-us-30GB
15.04
3.19.0-15
Ubuntu-15_04-amd64-server-20150422-en-us-30GB
SUSE
SLES 12
3.12.36-38.1
suse-sles-12-priority-v20150213
suse-sles-12-v20150213
CoreOS
584.0.0
3.18.4
CoreOS 584.0.0
CentOS6.5, 6.6, 7.0
LIS 4.0 Required 7.1
3.10.0-229.1.2.el7LIS 4.0 Recommended Oracle
6.4LIS 4.0 Required
7.0
Contact Support for details
Actual performance may vary depending on the
distro
, settings and file system
See more Linux instructions on Premium Storage document at azure.microsoft.comSlide38
Topics
Service Specifications
VM and Disk sizes and limits
IOPS and Throughput
Traffic Patterns
IO Size
Queue Depth
Cache hits
Migration
Cost Optimization
Migration tips
Backup/DR
Backing up the VHDs
Azure Backup
P
P
PSlide39
Backing up your DisksSnapshot the VHD blobs to take backups
Use Storage REST APICopy snapshots to a different storage accountRecommend using a Standard_GRS account for DRCopying snapshots require an intermediate copy blob in the same accountSlide40
Azure Backup serviceAzure Backup service for VM/Disk
Currently in PreviewSupport for Premium Storage is coming soonSlide41
Additional info…Analytics
Use the disk metrics available on PortalUse PerfMon, IOSTAT for additional detailsStorage REST API AccessNeeds Storage REST version 2014-02-14 and later
Public containers are not allowed – always use SAS
No REST access to mounted VHDs – use snapshotsSlide42
Topics
Service Specifications
VM and Disk sizes and limits
IOPS and Throughput
Traffic Patterns
IO Size
Queue Depth
Cache hits
Migration
Cost Optimization
Migration tips
Backup/DR
Backing up the VHDs
Azure Backup
P
P
P
PSlide43
Learn More
Premium Storage: High-Performance Storage for Azure Virtual Machine Workloads Premium Storage REST operationsUsing Blob Service Operations with Azure Premium Storage“DS” series VM specificationsMigrating to Azure Premium StorageChannel 9 video on Premium Storage
Azure Premium Storage Resources