vNext Syama Poluri Claus Joergensen Shai Ofek BRK3496 Brief History Microsoft SDS Past Present and Future Dells Journey with Storage Spaces Learnings from field and ID: 458284
Download Presentation The PPT/PDF document "Deploying Private Cloud Storage with Del..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1Slide2
Deploying Private Cloud Storage with Dell PowerEdge Servers and Windows Server vNext
Syama Poluri, Claus Joergensen , Shai Ofek
BRK3496Slide3
Brief History
Microsoft SDS : Past , Present and Future
Dell’s Journey with Storage
SpacesLearnings from field and Cloud Platform System (CPS)Dell Storage with Microsoft Storage SpacesStorage Spaces Direct Technology OverviewPhysical ViewLogical View I/O StackReFS Storage Spaces Direct DeploymentPowerEdge R730xd Platform DetailsManaging , Monitoring and TroubleshootingDell & Microsoft Partnership
AgendaSlide4
Microsoft SDS : Past , Present and Future
Dependent
on
Traditional Storage Relied on 3rd party SAN vendors1st wave SDSIntroduction to Storage Spaces “Remote File solution for Hyper-V using shared JBODIntroducied Scale Out File Server ( SOFS)RDMA and MultichannelSAN independence Storage Spaces Direct Remote File solution for Hyper-V using shared nothing hardwareUnlock use of DASReduce Cost Software Storage BusSATA and NVMeNode Fault Tolerance
Health Services
Storage Spaces Enhancements
Scalable
pools
Data Rebalancing
Pre Windows 2012
Windows 2012
Windows 2012 R2
Windows 2016
2nd wave SDS
Storage
Tiering
WriteBack
Cache (WBC)
Scale
Out File Server
improvements
SMB Direct Improvements
Volume Rebalancing
Multiple instances
JBOD monitoring Slide5
Dell’s Journey With Storage Spaces
Sept 2012: Windows 2012
Release:
Understanding the technology and feedback to Microsoft Oct 2013: Windows 2012 R2 ReleaseInternal testing and active feedback to Microsoft
Nov 2013:
Worked closely with Microsoft in developing JBOD monitoring feature
Feb 2014:
Support
Storage
S
paces
using PowerVault MD1200 and MD1220
May 2014
:
Support Storage
S
paces
using PowerVault
MD3060e
End-End
VDI solution using PowerVault MD1220
Sept 2014:
Additional
Hardware
Support with 12Gbps SAS . Select PowerEdge servers , PowerVault MD1400 and MD1420
Oct 2014
:
Launch
Cloud Platform System (CPS) Powered by Dell
Journey Continues…Slide6
Dell PowerEdge
servers
Dell Storage dense enclosures
Dell Networking switchesOptimized racking & cabling for high density, servicing, reliability & redundancyMicrosoft Cloud Platform SystemAzure-consistent Cloud in a BoxWindows Server 2012 R2, System Center 2012 R2, Windows
Azure Pack
Microsoft-designed
architecture based on Public Cloud learning
Optimized run-books for Microsoft
applications
Microsoft-led support & orchestrated updates
Visit
Microsoft
Cloud Platform booth to see full demo and
h
ardware Slide7
Learnings from the field and CPS
Planning real world deployments Capacity planning
Performance
planningNeed guidance more than simple deployment guidesSelecting the correct hardwareSome HDD scale better than othersHBA firmware , Network cards and driver information that are tested and qualified at scale Configuration complexities How many columns, HDDs and SSDsSAS and network cabling Pool configuration and RDMA configurations Validation with Windows featuresOptimal configuration of windows features Latest Hotfix / QFE informationChange management for Storage Spaces clustersSlide8
Announcing
Dell Storage
with Microsoft
Storage Spaces Slide9
o
Unable to generate color coded feedback. It would require additional programs/libraries to be installed on end user system. Still investigating solution. Work around is stating (X/X) systems are the same and present a warning when something is different.
o For validating specific version numbers, OS, KB, etc. when going to from release to release the scripts will either need to be hard coded (not ideal) or be able to take in a config file of some sort to check against (better solution but would require lifecycle control from release to release). · Current and future planso Improve the content gathered in the StorageInventory.ps1
§
Gather details on tiering configuration
§
Develop python script to examine details of content gathered
o
The end goal is to expand and improve the content created by each PowerShell Script and have a corresponding python module for each set of output files.
o
As each module gets more robust, the plan is to unify them into one or pass them on to create an actual GUI tool developed in Java/C++ (most likely done by someone full time).
Compute
Networking
Client I/O
Controllers
Storage I/O
Storage
Dell Storage with Microsoft Storage Spaces (DSMS)
More information at http://www.dell.com/microsoftSlide10
Private Cloud
Backup & archival
Validated & tested solution blocks
VDI
Hyper-V
Worldwide installation services and total solution support
SQL
SMB3 & RDMA
Sizing & performance guidance
Standard management with MS System Center,
iDRAC
,
OpenManage
Launching
June 2015
Dell Storage with Microsoft Storage Spaces (DSMS)Slide11
Sizing
Performance and Capacity sizing
SKUs for pre-defined configurations
Optimized solution SKUs using standard Dell servers and JBODsDeployment
Installation and deployment option Dell services
Validation assistance for deployment on-site
Comprehensive Deployment Guide
Management
Leverages Dell & Microsoft management tools
Solution updates with single payload for storage nodes and JBODs through Microsoft Cluster Aware Updating (CAU)
Support
Full solution-level support for all components
Dell worldwide support services
End-to-End Dell Engagement for lifecycle of entire solution
Dell Storage with Microsoft Storage Spaces Advantage
Visit
Dell
booth on Expo floor for demo and solution details Slide12
DSMS as a Building Block for Storage Solutions
Customer- a la carte
Sizing guidance & Best Practices
Deployment GuideBest-in-class componentsSystem Center integrationFast Track SolutionsValidated configurationSimple setupHigh availabilityEasy scalability
Reference Architectures
Verified configuration
Deployment guide
Scalable & Customizable
Partner & Dell Services
Best-in-class components
Professional expertise and custom solution development
Microsoft compatibility
System Center integration
Dell Storage with Microsoft Storage SpacesSlide13
Introducing Storage Spaces Direct Slide14
Storage Spaces Direct
Enabling cloud hardware designsSupport for DAS (shared nothing) storage hardware
Prescriptive
configurationsScale for Technical Preview 2Minimum of four serversMaximum of twelve servers Maximum of 240 disk devices in a single poolFault toleranceFault tolerance to disk, enclosure and node failures3-copy mirror and dual parityResilient up to two node failuresManagementSystem Center and PowerShellSimple expansionAdd node(s) and expand poolRebalance for capacityScale-Out File Server
Hyper-V Clusters
SMB3 Storage Network Fabric
Shared JBOD
Storage
Internal
DisksSlide15
Spaces Direct Logical View
Storage Access (Scale-Out File Server)Remote data access for Hyper-VData access resiliencyFile System (CSVFS/ReFS)System-wide data accessFast VHDX creation, grow and merge
Data Repair
Storage SpacesStorage poolingVirtual disksData storage resiliencySoftware Storage BusSpans all storage nodesVirtualizes physical disksLeverages SMB3 and SMB DirectStorage nodes with internal disksSATA, NVMe, SASSoftware Defined Storage SystemStorage NodeStorage Node
Storage Node
Storage Node
Storage Pool
Storage Space
Virtual
Disks
Scale-Out File
Server
\\
FileServer\Share
Cluster Shared
Volumes
C:\ClusterStorage
SMB3 Storage Network Fabric
Software Storage BusSlide16
File Server
File System
Clusport
Virtual HBA
CSVFS
VolMgr
PartMgr
Disk
SpacePort
PartMgr
Disk
HBA Miniport
ClusBT
Volume
Mgr
Write arrives from compute node
SMB sends it downstream
Server 1
Server 2
\\fileserver\share1
C:\clusterstorage\volume1
Virtual Disks
SBL Disks
Physical Disks
Spaceport sends write to each copy on disk
SBL forwards write to block target on destination node with shortest queue
SBL on each node sends data to physical stack
Access Layer
Virtualization Layer
Storage Bus Layer
Physical
Layer
File System
VolMgr
PartMgr
Disk
SpacePort
PartMgr
Disk
HBA Miniport
Spaces Direct I/O Stack
Clusport
Virtual HBA
ClusBT
Volume
MgrSlide17
ReFS is the Primary File System
VM checkpoints have parent-child VHDs
After backup completes, Hyper-V historically moved all data from child to parent (slow VHD “merge”)
Using ReFS, Hyper-V now quickly reassigns blocks from the child to the parentEfficientVM checkpoint clean-up after backup
Available per-file integrity streams validate file contents
Upon error detection,
ReFS
retrieves a correct copy from Storage Spaces, if available
ReFS
uses the correct copy to automatically repair the corruption
Resilient
Optional automatic error correction
HDDs are optimized for capacity
SSDs are optimized for performance
ReFS
keeps hot data in SSDs and cold data in HDDs, providing both performance and capacity
Affordable
Performance of SSDs, capacity of HDDs
SSD
Does not need to pre-fill new files and growing files with zeros
Quickly creates large fixed VHDs
Quickly grows dynamic VHDs
Accelerated
VHD creation and growthSlide18
ReFS: Resiliency
Metadata ChecksumsChecksums protect all filesystem metadataUser Data ChecksumsOptional checksums protect file dataChecksum Verification
Occurs on every read of checksum-protected data
And during background scrubbingHealing of Detected CorruptionHealing occurs as soon as corruption is detectedHealthy version searched for in Spaces alternate copies (i.e. mirrors or parity data)Corrupt version overwritten by healthy versionDisk 1Disk 2Disk 3Spaces 3-Way Mirror with ReFS
Reads striped across all three disks
Checksums verified on reads
On checksum mismatch, mirrors are consulted
Good copies used to heal bad mirror
BRK3474:
Enabling Private Cloud Storage Using Servers with Local DisksSlide19
Storage Spaces DirectHardware Deployment Options
Visit
Microsoft
Cloud Platform booth to see full demo and hardware Slide20
PowerEdge R730xd
Up to 2 Intel Xeon E5-2600 v3 (18 core)
CPUs
24DIMM slots(DDR4, 2133)Up to 16x3.5 “+2 x2.5“ Drive Slots64 TB for Data with 4TB disk 96 TB for Data with 6TB disk 2U HeightUp to 6 PCIe slots Network Select Adapter
(choice of onboard NIC)
Internal Storage ( SAS , SATA,
NVMe
Storage Spaces Certified HBASlide21
PowerEdge R730xd Overview (Rear)
2 x 2.5” Drive Bays
vFlash
Media Card SlotDedicated Management PortSerial ConnectorVideo Connector
USB 3.0 / 2.0 Ports
Network Daughter Card (choice of 4 x 1GbE or 2 x 10GbE + 2 x 1GbE); SFP+ Option Available
Up to 2 x Redundant Power Supplies – 495W, 750W and 1100W (AC or DC) options
Half Length, Low Profile Slots
PCIe3 x8 (x16 connector
)
Full Length, Full Height Slots
PCIe3 x16 (x16 connector
)
Full Length, Full Height Slot
PCIe3 x8 (x16 connector)Slide22
PowerEdge R730xd Overview (Internal Bay)Slide23
“Dell has hit a long
home run with the PowerEdge R730xd, delivering a
powerful 2U server
with lots of room to grow.” Exceptional performance and versatility for dynamic workloadsDell PowerEdge R730xd and R730 industry reviews
“The storage-centric
R730xd offers
new degrees of flexibility
…
to keep storage as close as possible to the compute layer.”
“The results are in, and the Dell PowerEdge R730
kicks some serious butt.
”
PowerEdge
R730xd offers an
“incredible range of configurability.”
Intel Technology Innovation Accelerated AwardSlide24
DEMO @ Microsoft Cloud Platform Booth
PowerEdge R730xd Spaces Direct Cluster
Storage : 13x 4TB HDD+4x900 GB SSD
Network : Intel 520 10GB+i350 GBRDMA : Mellanox Connectx-3 10GB SPFPowerEdge R630 Hyper-V Cluster Dell S4810 Networking Switch Slide25
Deployment ProcessSlide26
<-New
Simple deployment using PowerShell
#Create cluster
New-Cluster –Name cluster01 –Node machine1, machines2, machine3 –NoStorage# Enable Storage Spaces Shared Nothing (Get-Cluster).DASModeEnabled = 1#Create file serverAdd-ClusterScaleOutFileServerRole -Name sofs1#Create storage pool and define storage tiersNew-StoragePool -StorageSubSystemFriendlyName *clustered* -FriendlyName DAS -ProvisioningTypeDefault Fixed -ResiliencySettingNameDefault Mirror -PhysicalDisk (Get-PhysicalDisk | ? CanPool -eq $true)#Create virtual diskNew-Volume -StoragePoolFriendlyName DAS -FriendlyName vdisk01 -PhysicalDiskRedundancy 2 -FileSystem CSVFS_REFS –Size 4000GB#Create file sharemd c:\clusterstorage\volume1\share1 new-smbshare -name share1 -path c:\clusterstorage\volume1\share1 -fullaccess HyperV01, Hyper-V02set-smbpathacl -sharename share1Slide27
Monitoring and TroubleshootingSlide28
Dell OpenManage
Integration Suite for Microsoft
System Center
Dell Server and Client Deployment PacksDell Server and Client Update CatalogsDell Server PRO Management PackDell Hardware Management Pack SuitesDeploy
Update
Monitor
Virtual Management
Dell Lifecycle Controller IntegrationSlide29
Storage Diagnostic Logs Cmdlets
#Starts storage tracing sessions in the given subsystemStart-StorageDiagnosticLog -StorageSubSystemFriendlyName
“
RemoteSQLCluster”#Repro the issue#Stops currently running storage tracing sessions in the given subsystemStop-StorageDiagnosticLog -StorageSubSystemFriendlyName “RemoteSQLCluster”#Gathers storage event logs, trace logs & live dumps from each node in the clusterGet-StorageDiagnosticInfo -StorageSubSystemFriendlyName “RemoteSQLCluster” –DestinationPath D:\StorageLogs\Slide30
SCOM Storage Monitoring
Storage subsystem determines healthFocus on relevant objectsStorage subsystem, volumes and file sharesAutomatic remediation when
possible
Actionable alertsAlert specifies urgencyAlert specifies remediation actionAlerts automatically resolve when issue is addressedEasy to determine affected objectsConsumable through PowerShell or 3rd partiesSMAPI Storage SystemActions
System Center
vNext
Health Logic
Operations Manager
Dashboard
User notifications
3
rd
Party
Monitoring
PowerShell
Virtual Machine Manager
Discovery
ActionsSlide31
Data Center Networking
Dell Management Software and Plug-ins for System Center
Scale-out file server clusters
Storage Space
Storage Space
Storage Space
Hyper-V clusters
Dell PowerVault JBOD Storage
Dell PowerEdge Servers
Dell Network Switches
Dell PowerEdge Servers
Dell & Microsoft Key Value
Dell-Microsoft End –End Solutions (VDI,Private Cloud)
Tested and Validated for Scalable and
Reliable DeploymentSlide32
Storage Spaces (Windows Server 2012 R2)Microsoft and Dell validated Hardware and solutions
Dell Storage with Microsoft Storage Spaces Cloud Platform SystemBuilt on Dell HardwareBest Practices, Driver optimization, etc.Storage Spaces Direct (Windows Server vNext)Co-Engineer Solution
Joined TAP
Dedicated Engineering teams: Microsoft and DellDell and Microsoft PartnershipContinuous and Co-EngineeringSlide33
Leverage the power of Dell and Microsoft alliance
Joint IP/development
Engineering labs, technology centers, centers of excellence
US Windows Deployment Partner of the Year
2 years running
Most complete
Full, end-to-end capabilities – from tablets to the data center
Global Solutions Centers
Demonstrate the full Microsoft stack
running on Dell
Single Point of Contact
Dell provides a single contact for total solution
TAP
participation
Broad
participation in Technology Adoption Programs
US Windows Partner of the Year
2013Slide34
“Dell is Microsoft’s largest OEM partner to have implemented support for Storage Spaces.
Dell provides the most complete Storage Spaces implementation – server, controller, JBOD and disks form an end-to-end solution, and this support includes new features such as JBOD monitoring.”
Microsoft Server and Cloud Platform Team
TechNet Server & Cloud BlogSlide35
Storage Spaces (Windows Server 2012 R2)Scalable , Resilient and Efficient SDS solution
Dell and Microsoft collaborate , innovate and optimize for real worked deployments For today’s Storage Spaces needs deploy Dell DSMS solutionsVisit Dell’s Booth on Expo Floor Storage Spaces Direct (Windows Server vNext)Join TAP
Visit Microsoft Cloud Platform Dell Booth in Expo
Floor Visit www.delltechcenter.com for updated informationLeverage the power of Dell and Microsoft allianceSummarySlide36
Q & ASlide37
Visit
Myignite
at
http://myignite.microsoft.com
or download and use the
Ignite Mobile App
with the QR code above.
Please evaluate this session
Your feedback is important to us!Slide38