Ram Poornalingam and William Rall - PowerPoint Presentation

Slide1
Slide2

Ram Poornalingam and William RallProgram Manager and DeveloperMicrosoft

Exchange Online Migrations Technical Deep Dive

DMI305Slide3

Session Objective And Takeaways

Understand Migration optionsDeep dive on Hybrid MovesTroubleshoot Common Issues

Performance

Takeaway

Office

365 has a simple set of migration solutions that you can useSlide4

Read case studies

and documentation

1. Plan

2. Prepare

Add and Verify SMTP domains

DNS Administration

Deployment Plan

Configure On Premise

3. Migrate

4. DecommissionSlide5

MigrationThird Party

IMAP migration

Exchange

Cutover

Exchange migration

Staged Exchange migration

Hybrid migration

IMAP migration

Cutover migration

Staged migration

Hybrid

Exchange 5.5

X

Exchange 2000

X

Exchange 2003

X

X

X

X*

Exchange

2007

X

X

X

X

Exchange

2010

X

X

X

Exchange

2013

X

XXNotes/DominoXGroupWiseXOtherX

* Must have Exchange 2010 hybrid server on-premises, cannot use 2013 hybridAdditional options available with tools from migration partners

Migration OptionSlide6

How to pick a Solution?

1 150 5,000 25,000

Organizational Size in Users

C-EM

S-EM

Hybrid

Migration Solutions

<1 Week 2 Weeks 3 Weeks Several Months

Hybrid

None Mailflow/GalSync Free/Busy, Archive in Cloud

Time For Migration including PlanningSlide7

Deployment Assistant DemoSlide8

Hybrid MRS MovesSlide9

Introduction

Batch MovesRich Web Management Manage

collection

of

moves

Protocol

AgnosticSource server throttling Email Reporting Slide10

Lifecycle of Migration

Step 1 Create Migration Endpoint

Step 2

Create and Start Migration Batches

Step 3

Review Report

Step 4 Complete BatchesSlide11

A endpoint configuration management mechanism

Captures the remote server informationThe required credentials for migrating the contentSource Throttling Setting

An endpoint has a type associated with it

We support 3

OutlookAnywhere

ExchangeRemote

IMAPMigrationEndpoint are only applicable to Cross-Forest Remote and Onboarding moves.New-MigrationEndpoint

is the command-let usedLifecycle of MigrationStep 1 Create Migration Endpoint

E15 FE

Office 365Slide12

Lifecycle of MigrationStep 1

Create Migration Endpoint

$

ep

=

New-

MigrationEndpoint

–Name consotoEndpoint –

ExchangeRemoteMove

–AutoDiscover –EmailAddress rampo@consoto.com –Credential $credsSlide13

Lifecycle of Migration

Step

2

Create

Migration Batch

New-

MigrationBatch –Name OnboardingBatch1 –

SourceEndpoint $ep.Identity –TargetDeliveryDomain contoso.mail.onmicrosoft.com –CSVData

$dataSlide14

Initial Data Sync

Incremental Sync

Completion

Lifecycle

of Migration

Step 2

Create Migration

BatchSlide15

Lifecycle

of Migration

Step 3

Review

ReportSlide16

Automatic Completion

Manual Completion

Like

New-

MoveRequest

-

SuspendWhenReadyToCompleteWe will do a throttled Resume-

MoveRequest every 24 hrs.Ideal for onboarding/cross-forest moves over weekend/offhours

Quick switch over for the individual account

Finalization for the batch as whole is spread over a large period of timeIdeal for local moves

Initial Sync

Inc Sync

Completion

Initial Sync

Inc Sync

Completion

Lifecycle

of Migration

Step 4

Complete

MigrationsSlide17

MigrationUser

Get-MigrationUserGet-

MigrationUserStatistics

MigrationBatch

New-

MigrationBatchStop-MigrationBatchStart-

MigrationBatchRemove-MigrationBatchSet-MigrationBatch

MigrationEndpoint

New-

MigrationEndpoint

Set-

MigrationEndpoint

Remove-MigrationEndpointTest-MigrationServerAvailability

MigrationStatistics

Get-

MigrationStatistics

Management Features

Common

Command-let

ManagementSlide18

Hybrid Move DemoSlide19

ArchitectureSlide20

Office 365

E2010 CAS

E2010 or 2007 MBX

On-premise

Migration Service

Mailbox Replication Service

Mailbox Replication Service

Mailbox Replication Service

Active

Directory

2

1

4

4

5

Step1

Admin creates a batch of users using New-

MigrationBatch

Step2

Migration Service injects move requests to multiple MRS service’s queue

Step3

For each request,

MRS service initiates mailbox move with MRS proxy

Step 4.1

For each request, MRS proxy on source reads data from the mailbox over MAPI and transmits to MRS over https

Step 4.2

MRS service writes this data into target using XSO protocol

Step 5

MRS finalizes the move and converts mailbox on the source to remote mailbox .

Step 6

At this point clients and

mailflow

is redirected

3

4HTTPSXSOMAPIArchitecture2010 to CloudSlide21

Office 365

On-premise

Migration Service

Mailbox Replication Service

Mailbox Replication Service

Mailbox Replication Service

2

1

Step1

Admin creates a batch of users using New-

MigrationBatch

Step2

Migration Service injects move requests to multiple MRS service’s queue

Step3

For each request,

MRS service initiates mailbox move with MRS proxy via FE machine

Step 4.1

For each request, MRS proxy on source reads data from the mailbox over MAPI for E2010 and XSO for E2013 and transmits to MRS over https

Step 4.2

MRS service writes this data into target using XSO protocol

Step 5

MRS finalizes the move and converts mailbox on the source to remote mailbox .

Step 6

At this point clients and

mailflow

is redirected

XSO

Architecture

2013 to Cloud

E2013 FE

E2010 or

2007Mbx

3

5

4HTTPSE2013 BE3HTTPMAPISlide22

TroubleshootingSlide23

Scenarios

Connectivity IssuesMoves are FailingContent Not MigratedPerformanceSlide24

Troubleshooting

Common issues during preparation :Bad certificatesBad passwords

Insufficient credentials

Protocol disabled

Firewalls blocking ports

Load Balancers reshaping trafficSlide25

Troubleshooting : Connectivity Issues

AutoDiscover

use testexchangeconnectivity.com to diagnose

AutoD

IssuesSlide26

Troubleshooting : Connectivity Issues

AutoDiscover

C:\> Test-

MigrationServerAvailability

–

ExchangeRemoteMove

–RemoteServer x –Credential $

badCred

-AutodiscoverSlide27

Troubleshooting:

AutoDiscover

testing

testexchangeconnectivity.comSlide28

Troubleshooting : Connectivity Issues

Connectivity General ErrorSlide29

C:\> Test-

MigrationServerAvailability

–

ExchangeRemoteMove

–

RemoteServer

x –Credential $

badCred

Http request is unauthorized with client authentication scheme ‘Negotiate’………(401) Unauthorized.

 The logon attempt failed.

Troubleshooting : Connectivity IssuesBad CredentialsSlide30

Troubleshooting : Connectivity Issues

Bad Certificates

C:\> Test-

MigrationServerAvilability

–

ExchangeRemoteMove

–

RemoteServer x –Credential $badCert

Could not establish trust relationship for the SSL/TLS secure channel …….remote certificate is invalidSlide31

Troubleshooting : Connectivity Issues

MRS Proxy Disabled

C:\> Test-

MigrationServerAvilability

–

ExchangeRemoteMove

–

RemoteServer x –Credential $goodCred..............The remote server returned an error: (403) ForbiddenSlide32

Moves are FailingSlide33

Common issues:Permissions issues (password changes)Bad/Large Item limit exceeded

Issues with load balancers, firewalls, etc.

Troubleshooting : Move FailuresSlide34

Troubleshooting : Move Failures

InvestigationSlide35

Troubleshooting : Move Failures

InvestigationSlide36

Troubleshooting : Move Failures

ReportsSlide37

Troubleshooting : Move Failures

Investigation -PowerShellSlide38

Troubleshooting : Move Failures

Password ExpiredSlide39

Content Did Not MigrateSlide40

INBOX

INBOX

Troubleshooting: User Content Not Migrated

Large Messages

Skipped Item Count +=1

Source Mailbox

Target Mailbox

Max Email Message size is

35

MBSlide41

Troubleshooting: User Content Not MigratedBad Messages

INBOX

INBOX

Source Mailbox

Target Mailbox

Skipped Item Count +=1Slide42

Troubleshooting: User Content Not MigratedSlide43

Troubleshooting: User Content Not MigratedSlide44

Troubleshooting: User Content Not MigratedSlide45

SymptomError -

CommunicationError with “invalid sequence number” error message

Reason

SessionAffinity

is broken due various reasons like Load Balancing

Solution

Client session affinity is requiredExchangeCookie is a good approach. It is stamped by MRSProxy in first reply

Client IP affinity is not good: all connections out of DC appear to come from the same IP; you don’t want to overload a single CAS box.Troubleshooting: Load Balancer IssuesSlide46

PerformanceSlide47

Performance

Why is Move Queued?Why is Move Slow?Slide48

PerformanceQueued Duration

95% of requests are picked up under 15 minutesSlide49

PerformanceThroughput

Low throughput due to transient issues – typically due to high latencySlide50

C:\> . .\

AnalyzeMoveRequestStats.ps1

C:\>

$

stats = $moves | Get-

MoveRequestStatistics

–IncludeReportC:\> ProcessStats

-stats $stats -name ProcessedStats1 StartTime :2/18/2014

19:57

EndTime :3/3/2014 17:15 MigrationDuration :12 day(s) 19:10:55 MailboxCount

:50 TotalGBTransferred :2.42 PercentComplete

:95 MaxPerMoveTransferRateGBPerHour :1.11

MinPerMoveTransferRateGBPerHour :0.43 AvgPerMoveTransferRateGBPerHour

:0.66 MoveEfficiencyPercent

:86.36 AverageSourceLatency :123.55 AverageDestinationLatency :

IdleDuration :1.16% SourceSideDuration :78.93

%

DestinationSideDuration  :19.30%

WordBreakingDuration :9.63%

TransientFailureDurations :0.00% OverallStallDurations :4.55% ContentIndexingStalls :1.23%

HighAvailabilityStalls :0.00% TargetCPUStalls :3.32% SourceCPUStalls

:0.00% MailboxLockedStall

:0.00% ProxyUnknownStall

:0.00%http://blogs.technet.com/b/exchange/archive/2014/03/24/mailbox-migration-performance-analysis.aspx

Performance

AnalysisSlide51

Performance

Increase

MaxConcurrentMigrations

Best way to increase effective throughput is increase concurrency

Protect source servers

Max of 100 to ensure fairnessSlide52

Performance

Increase Buffer size

Beneficial for high latency networks >100ms

Beneficial when size >10mb

Only supported in E15 CU3

<

MRSConfiguration

…

ExportBufferSizeOverrideKB

="7500"

Able to pack more message per chunk

20 %

20 - 90%

Source Enumeration

Initial Data Sync

Incremental Sync

Finalization

90%

95%

Reduces this duration

Reduces this durationSlide53

Performance

Distribute load on source

Author batch with mailbox

from

multiple

source

serversAdd capacity on sourceSlide54

What’s NextSlide55

What’s Next ?

Simple Migration SchedulesUnify Exchange Migration SolutionsImprove IMAP error reportingSlide56

Session Objective And Takeaways

Office 365 has a simple set of migration solutions that you can useSlide57

Enterprise: Investigating MRS not making progress (long-Queued Moves)

$targetDB

= $(Get-

MoveRequest

username).

TargetDatabase

$server = $(Get-MailboxDatabase -Status $targetDB).MountedOnServerGet-ExchangeDiagnosticInfo -Server $server -Process

MSExchangeMailboxReplication -Component MailboxReplicationService -Argument "PickupResults,Queues=$targetDB" | fl Result Result      : <Diagnostics>                <

ProcessInfo>

                  <id>21132</id>                  <serverName>...</serverName>                  <startTime>2014-02-26T01:23:54.2838965Z</startTime>                  <currentTime

>2014-02-27T21:50:36.9212182Z</currentTime>                  <lifetime>1.20:26:42.6373217</lifetime>                  <threadCount>..</threadCount>

                  <handleCount>..</handleCount>                  <workingSet>..</

workingSet>                </ProcessInfo

>                <Components>                  <MailboxReplicationService

>                    <ServiceStartTime>2014-02-26T01:23:59.3462951Z</ServiceStartTime>                    <LastScanDuration>1484</LastScanDuration>                    <LastScanTime>2/27/2014 9:47:32 PM</LastScanTime>

                    <DurationSinceLastScan>184574</DurationSinceLastScan>                    <NextFullScanTime>2/27/2014 10:03:12 PM</NextFullScanTime>

                    <NextLightJobsFullScanTime

>2/27/2014 9:59:25 PM</NextLightJobsFullScanTime>                    <Queues>

                      <Queue Guid="47bf9a71-d40d-4cac-8645-77264d569a03" DBName="targetDB“ LastJobPickup="2014-02-27T21:50:23.7965459Z"

LastInteractiveJobPickup="0001-01-01T00:00:00“ QueuedJobs="0" InProgressJobs="1" MdbDiscovery="2014-02-26T01:36:20.2551108Z“ LastScan="2014-02-27T21:50:22.9528187Z" LastScanDurationMs="874" NextScan="2014-02-27T21:50:33.8587918Z“ lastfinishtime="2014-02-27T21:50:33.8587918Z" lastfinishjob="4f3f52bb-a2cb-4b00-97b5-ead27126ce3f">

                        <LastScanResults>

                          <JobPickupRec RequestGuid="4f3f52bb-a2cb-4b00-97b5-ead27126ce3f" Type="Move“ Status="InProgress" Workload="Local" Pri="50" LastUpdate="2014-02-27T21:50:21.8434617Z“ PickupAttemptTime="2014-02-27T21:50:23.7965459Z" Result="JobPickedUp“ NextRecommendPickup

="9999-12-31T23:59:59.9999999">SyncStage IncrementalSync, CancelRequest=False, Priority=Normal</JobPickupRec

>                        </LastScanResults>

                      </Queue>                    </Queues>                  </MailboxReplicationService>                </Components>

              </Diagnostics>Slide58

OutlookAnywhere bad credentialsSlide59

SupportabilityGet-MigrationBatch

-IncludeReportSlide60
Slide61

©

2014

Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.