Veeam Backup amp Replication Tips and Tricks Anton Gostev Veeam Software Gostev Doug Hazelman Veeam Software VMDoug vmworldsponsor Quick Overview of v6 Architecture ID: 463891
Download Presentation The PPT/PDF document "SPO3292" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
SPO3292
Veeam
Backup & Replication:Tips and Tricks
Anton GostevVeeam Software@GostevDoug HazelmanVeeam Software@VMDoug
#
vmworldsponsorSlide2
Quick Overview of v6 Architecture
Backup serversBackup proxy serversBackup repositories
“Automated everything”Intelligent load balancingCentralized management via Enterprise ManagerSlide3Slide4
Scaling your backups
3 simple rulesSlide5
Scaling your backups
Keep the management server happyDisable default proxy (requires
6.1 or later)Allocate enough RAM for job manager processesKeep concurrent (running) jobs under 100 per management serverBackup proxy servers: the more, the better?No! Too much load on storage and networkUse max concurrent tasks limit on proxies or repositoriesWatch for “job timed out waiting for resources” messagesBackup repository considerationsBe careful with the reversed incremental backup mode!Limit concurrent jobs on backup repositories to a reasonable amountUse ingest rate throttling for cross-SAN backupsSlide6
A word on backup repositories
Don’t underestimate the importance of performance!By far, the most commonly reported bottleneck
What makes the best backup repositories?Windows or Linux server (can be same as backup proxy server)Local storage, DAS or SAN mounted for physical serverpRDM disk (vSphere 5+), or iSCSI LUN connected via in-guest iSCSI for virtual serverSub-optimal backup storageNAS or network shareVMDK on VMFS (size and recoverability considerations)RAID levelIf you can afford it, use RAID10 (again, performance)Slide7
Processing Modes
All you need to knowSlide8
Direct SAN Access: The good
Fastest processing modeLeast impact on production
Backup processing is fully offloaded to dedicated backup proxy serversBackup traffic is isolated to the storage network (aka LAN-free)Does not impact consolidation ratio, so cheapest tooSlide9
Direct SAN Access: The bad
Supports block storage onlyFC (fibre
channel): physical backup proxy server only!iSCSI: physical and virtual backup proxy servers both supportedPhysical backup proxy server requirement for FC SANMight not go along well with your virtualization projectConsider repurposing older serversMight be hard for beginners to setupSee Veeam Forums FAQ for step-by-step guideManual datastore mapping might be requiredFor certain SANs, B&R may not able to detect proxy connectivitySlide10
Direct SAN Access: The ugly
What’s the worst that could happen?Windows re-signaturing your VMFS LUNs!
vSphere will no longer recognize datastoresDon’t panic, VMware Support should be able to fixThree easy ways to get into troubleWindows Explorer automounting new volumes (but not with Veeam)Clicking Disk Management snap-in popup without readingGiving Local Administrator rights to random peopleSlide11
Direct SAN Access: The safe way
Present VMFS LUNs to backup proxy server as read-only
Most SANs support it these days—chase your vendor if yours does notDisable automount on your backup proxy serversDo it the right way: use SANPolicy Windows setting!Veeam backup proxy server setup does this automatically for youDisable Disk Management snap-in with Group PolicyUser Configuration > Administrative TemplatesWindow Components > Microsoft Management Console >Restricted/Permitted snap-ins > Disk ManagementKeep Local Administrator rights on backup proxy servers to yourselfCannot really do this for default proxy due to FLR requirementAnother reason to use dedicated backup proxy server!Slide12
Direct SAN Access: Tips & tricks
Got a fast SAN? Get a modern backup proxy server!Multi-core CPU (compression) and
fast RAM (inline deduplication)Update firmware and drivers across the boardDisabling MPIO may increase performanceiSCSI SAN? Tweak TCP/IP on backup proxy netsh interface tcp set global autotuning level = disableIncrease read-ahead bufferDefault is 4MB (optimal setting for most SANs)To change, create the new value in bytes: VddkPreReadBufferSize (DWORD) Slide13
Hot Add: The good
Easy to setup—very little planning involvedAny Windows VM can be made a Hot Add backup proxy
Fast data transfers with any storageDirect storage access (albeit through ESXi storage stack)Supports all types of storage (including NFS)Shared storage: at least 1 backup proxy server per vSphere clusterLocal storage or DAS: at least 1 backup proxy server per hostUse your existing Windows VMs (save on licensing)Data processing engine process runs with lower priority (6.1)Further CPU usage reduction in 6.5Allows for 100% virtual deploymentSlide14
Hot Add: The bad
Not as mature as other modesAffects your consolidation ratio
Backup proxy servers take host resourcesUltimately means more ESXi hosts, and more VMware licensesHot Add process itself is slooowCan take up to 1–2 minutes to complete for each VM—adds up quickly!Hot Add as a vSphere feature has a number of limitationsGood news—many are being removed as VDDK maturesSee FAQ on Veeam forums FAQ for the complete listSlide15
Hot Add: The ugly
Snapshot removal problems due to locks
Veeam B&R: multiple hooks in place to work aroundCBT must be disabled on backup proxy VMPrevents stun on Hot Add due to CBT initializationNFS-specific issueExtended VM stun on hot remove in some scenariosSlide16
Hot Add: Tips & tricks
Add extra virtual SCSI controller to backup
proxy serverA single SCSI controller can have a maximum of 16 disks attachedConcurrent jobs on the same backup proxy server can result in more!Keep vSphere and Veeam up to dateSingle block size in VMFS5 removes the most common hot add issueLatest Veeam Backup & Replication will have latest and greatest VDDK versionTry increasing read-ahead bufferSeems to really help with certain NFS storageAvoid cloning backup proxy VM For example, to provision additional backup proxiesSlide17
Network (NBD): The good
Easy to setup—in fact, no setup is requiredAny existing server (physical or virtual) would do
Supports all types of storage, including NFSServer placement does not matter (unlike with Hot Add)Very quick to initialize data transferCan be quite fast—with 10Gb EthernetSlide18
Network (NBD): The ugly
Painfully slow performance on 1Gb EthernetAverage speed reported is 10-20 MB/s
Leverages ESXi management interfaceSlide19
Network (NBD): Tips & tricks
1 Gb EthernetUse for sites with low change rate
Works faster than other processing modes in such conditionsKeep at least one Hot Add backup proxy server aroundFull VM and virtual disk restores take forever over NBDKeep in mind intelligent load balancing algorithmsNetwork backup proxy servers have lowest priority!Upgrade to 6.1 or laterImproved network proxy location awarenessSlide20
One last thing
This hack significantly reduces supportability!Cut up to 5 minutes of processing time
per VM by disabling VDDK loggingApply in stable environments only!Create the new value and set to 1: DisableVDDKNetworkOutput (DWORD) Slide21
Deduplicating Storage
Yes, you can afford it!Slide22
Deduplicating storage: The good
What gives? Global dedupe!Deduplication across backup files from different jobs
Perfect for long-term backup archivalTop hardware appliances among Veeam usersEMC DataDomainExaGridHP StoreOnceTop software appliances among Veeam usersZFS-based appliancesStarWindWindows Server 2012 dedupe is awesomeSlide23
Deduplicating storage: The bad
Hardware appliances are expensiveAlthough they do provide excellent
dedupe ratioSoftware appliances are resource hogs Both performance and dedupe ratio are sub-par, tooWindows Server 2012 dedupe is awesomeIncluded free of charge—start using it today!Provides very decent dedupe ratioSlide24
Deduplicating storage: The ugly
Random access performance is lackingA problem for all
solutions featuring inline deduplicationTypically insufficient out-of-the-box for large-scale vPower usageException: post-process deduplicationExaGridRaw disk landing zone (full-speed vPower from recent backups)Veeam-specific logic further optimizes performanceWindows Server 2012 Backups “land” on raw storage at full speedOnly old backup files are deduplicated—great for vPowerDecent speed even off already deduped backupsSlide25
Deduplicating storage: Tips & tricks
Already own storage with inline deduplication?Inline data “rehydration” process is what makes vPower slow
Reduce the block size in Advanced job settingsWAN (256KB) and LAN (512KB)Reduced block size might impact backup performanceUse Linux-based backup repositoryLarge client cache, or even caching client file system (FS-cache), can significantly improve vPower performanceSlide26
Deduplicating storage: Tips & tricks (continued)
Keep Veeam
dedupe onUse incremental backup modeChoice of synthetic or active fulls depending on actual storageFor best dedupe ratio on device side…Disable compression (significantly increases amount of data transferred from backup proxy server to backup repository over network)For best backup performance and smallest windowKeep compression at defaultIf you like to avoid extremes…Set compression to Low (dedupe-friendly)Slide27
Deduplicating storage: Tips & tricks (continued)
Got more than one deduplicating
storage device?Use internal replication to sync backups offsite!Extremely traffic-efficient approachMany customers use and report great success!Keep the backups imported for easy DRTo automate repository refresh in DR site, use: Get-VBRBackupRepository -Name "DR_Repository" | Sync-VBRBackupRepositorySlide28
WAN Accelerators
Your WAN on steroidsSlide29
WAN accelerators: The good
Two types of WAN acceleratorsCaching WAN accelerators provide
significant bandwidth savings with Veeam replication, but are typically quite expensiveTransport layer WAN optimizers are unlikely to offer significant bandwidth savings with Veeam replication, and are usually cheapBoth improve reliability of TCPLong distance wireless or satellite linksIPsec rekey operations on a VPN tunnelWAN links with high jitter, packet loss or occasional dropsBoth allow long-running jobs to finish more consistentlyFor example, initial replication over networkBoth improve WAN utilization for most workloadsSlide30
WAN accelerators: What’s hot?
Top caching WAN accelerators among Veeam usersCisco WAAS
RiverbedSilverPeakTop WAN optimizers among Veeam usersHyper-IPSlide31
WAN optimizers: Tips & tricks
Veeam Backup & Replication leaves little room for bandwidth reduction by WAN optimizers
Built-in WAN optimizations in v6:Multiple TCP streams to maximize throughputNetwork traffic compression Consider using on unreliable networks, but don’t expect them to add you extra bandwidthUpdate to Backup & Replication 6.1 Patch 1 before evaluatingVeeam Backup & Replication not using all available bandwidth?Increase the amount of TCP streams (default is 5) DownloadStreamsNumber (DWORD)Slide32
Caching WAN accelerators:
Tips & tricksMultiple
TCP streams can cause issues!Disable multiple streams in B&R traffic throttling settingsDisable network traffic compression in VeeamLow (dedupe-friendly) compression level might be a better optionIf required, have network admins configure bypass on Veeam backup proxy servers to avoid polluting cachesSlide33
Veeam Backup & Replication 6.5
We never stand stillSlide34
What’s coming in 6.5
Veeam Explorer for Microsoft ExchangeVeeam Explorer for SAN Snapshots
VMware vSphere 5.1 supportWindows Server 2012 supportSlide35
Veeam Explorer for Microsoft Exchange
Visibility into Exchange VM backupsImmediate: No need to provision storage, restore the VM or
restore the mailbox storeAgentlessRequires no special backups or metadata collection—even works with existing Veeam backups (and SAN snapshots)Free!Included in all versions of Veeam Backup & Replication 6.5,including Free EditionEliminates need for expensive standalone tools licensed per-mailboxCurrently available in “exclusive beta”Just restored a 145 GB #MSExchange Public Folder database in 2 min. using @veeam. Then restored a single item all under 10 min. SWEET!The new #veeam explorer for exchange looks veeamy. That's right, I just made up a new word (it means awesomesauce)Really excited with the new @veeam Explorer beta for #Microsoft Exchange VM backups - we have lots of interested customers ready for this! Even the *beta* of @
veeam Exchange Explorer works a treat. Saved literally, hours of work.. and saved my bacon. #recommendSlide36
Veeam Explorer for Microsoft Exchange (continued)
CapabilitiesBrowse: familiar Explorer-type interface
Search: familiar Outlook-like Find, including Advanced FindExport: export to PST file, MSG file or attachmentUses casesE-discoveryItem-level restore: export and send to affected userMailbox archiveSupports Exchange Server 2010Slide37
Veeam Explorer for SAN Snapshots
Veeam restores from SAN snapshotsSupports tiered data protection strategy
Perform all restores through familiar, easy-to-use Veeam interfaceSupports HP StoreVirtual VSA and HP LeftHandSAN snapshots + Veeam restore = Best RPOs and RTOs for operational recoverySlide38
Veeam Explorer for SAN Snapshots (continued)
Fast: recover entire VM or individual items in < 2 minutes
Fully automated: clone & promote snapshot, present to vSphere, clean upRestores directly from VM files on the SAN snapshot: no staging or intermediate restores requiredFlexibleSpecific VMIndividual guest files: Windows, Linux, et alIndividual Microsoft Exchange itemsFreeWorry-free: automated process eliminates human errors and protects integrity of SAN snapshots and production LUNSAgent-free: no agents to deploy on hosts or VMsLiterally free: included in all editions of Veeam Backup & Replication 6.5, including Free EditionSlide39
Questions? Comments?Slide40
Thanks for attending!Slide41
FILL OUT
A SURVEY
AT WWW.VMWORLD.COM/MOBILECOMPLETE THE SURVEY WITHIN ONE HOUR AFTER EACH SESSION AND YOU WILL BE ENTERED INTO A DRAW FOR A GIFT FROM THE VMWARE COMPANY STORESlide42
SPO3292
Veeam
Backup & Replication:Tips and Tricks
Anton GostevVeeam Software@GostevDoug HazelmanVeeam Software@VMDoug#vmworldsponsor