#mail migration throughputs

[o365info-blog]

New Post has been published on o365info.com

New Post has been published on http://o365info.com/mail-migration-office-365-mail-migration-methods-part-14/

Mail migration to Office 365 | Mail Migration methods | Part 1/4

The subject of: mail migration to Office 365 includes many aspects such as:

Choose the suitable migration plan\method, based on the organization mail infrastructure organization requirement and so on.

Create a mail migration plan

Create a mail migration pilot

Estimate the mail migration throughput – Provide to the organization management an estimate of the resources that are required for the mail migration project and estimation of the time that it will take to complete the organization mail infrastructure to Office 365 (Exchange Online).

The point of this article

In the current article, we will review optional mail migration tools\methods that we can use for implementing mail migration to Office 365, and the focus is on the way that each of this mail migration methods uses for migrating the data to Exchange Online.

The information about the different methods doesn’t include a “How to” instruction and doesn’t include a comprehensive review on each of the features for the mail migration option but instead, focus on the subject of “how is the data migrated” when using different mail migration methods.

Office 365 – Mail migration technology

A quick reference for the article series

Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 The second article includes a review the different factors that impact the performance of the mail migration throughput. Mail migration to Office 365| Optimizing the Mail Migration throughput | Part 3/4 The third article recommendation and best practices for improving and optimize the performance of the mail migration throughput. Mail migration to Office 365 | Measure and estimate Mail Migration throughputs | Part 4/4 The last article in this series is dedicated to information about the “expected mail migration throughput” and includes a nice Excel based utility (Office 365 – Multiple and Single Mailbox Migration throughput calculator) that will help us to provide an estimation for the expected data transfer rate. Based on this information we can provide a reasonable “end date” for the completion of the mail migration project.

Mail migration to Office 365 | Methods and classifications

There are many available options that we can choose from for accomplishing the task of: mail migration to Office 365.

Microsoft Office 365 built-in mail migration tools

Third party mail migration tools

PST migration

Mail migration to Office 365 – Methods and classifications

Mail migration method Server to server versus Client to server

Method 1: Mail migration “server to Server”

An example for the Mail migration that I describe as: “server to Server” is the Office 365 built-in mail migration tools that are based on a “server to server” mail migration concepts in which the Exchange Online is the component that connects to the “other side” (the organization’s mail server) and “pull out” the data from it.

The additional basic concept is that the “destination server” the mail server that hosts the mailboxes is Exchange server.

Office 365 built-in mail migration tools – Server to Server method

The exception to the rule that the “destination server” is an Exchange server is the option of IMAP mail migration. The IMAP migration was designed for scenarios in which the organization’s mail server is not an Exchange server, but instead other mail product that can enable to “pull” the data (user’s mailboxes) by using the IMAP protocol. For example, when we want to migrate, a mail infrastructure that resides at Gmail, Google Apps or another kind of mail provider.

Note – although we can use IMAP protocol for performing mail migration from Exchange server, this option is not recommended because the IMAP protocol is very limited and basic, and he doesn’t know how “deal” with many types of data that exists in the Exchange user mailbox. My opinion is that the only time we will use the option of “IMAP mail migration” is only when we don’t have any other choice. For example: implementing a mail migration project when the “source mail server” is not Exchange server, but instead a “standard mail server” and the only option for “pulling the data” from this server is by using the IMAP protocol.

Method 2: “Client to Server” (Push migration)

The “Client mail Migration” is based on a concept in which the “source” (client or other source that has the mailbox data) is “pushing” the data into the Exchange Online mailbox.

An example for this method could be migration that describes as “PST migration”. When using this option the user is “responsible” for exporting the mailbox data to a PST file and then move\copy the data to the Exchange Online mailbox.

Additional example could be – the Microsoft tool: PST Capture that holds many users PST files in a central location and then can “push” the data to Exchange Online.

Some of the third party mail migration tools are based on this concept. The mailbox data is copied from the organization’s mail server into a central location (the migration stations) and “push” by the migration station to Exchange Online.

1. Office 365 built-in migration tools

Office 365 includes built-in mail migration tools (Native Office 365 mail migration tools) that was designed for different needs and different organization mail infrastructure.

Note – the term “Migration to Office 365” relates to additional aspects such as user migration, but, in this article, we relate only to the part of “mail migration”

The Office 365 built-in mail migration tools are:

Cutover migration

Stage migration

IMAP Migration

Hybrid migration

Native Office 365 mail migration tools | basic concepts

In the following section, I would like to review some basic terms\concepts that relate to mail migration when using the Native Office 365 mail migration tools.

Public presence

To be able to use the native Office 365 mail migration tools, the Exchange on-Premise server must have a “public presence” (Public IP, Public certificate, etc.). An additional requirement is that the Exchange on-Premise server will be configured for providing an Outlook AnyWhere service (the former term is RPC/HTTPS).

The need for “Public presence” is a mandatory requirement because when using the Native Office 365 mail migration tools, the mail migration process is implemented by the Exchange Online that “build” a communication channel to the Exchange Online that use for the mail migration process.

The “Public presence” of Exchange on-Premises server is needed for additional scenarios such as Hybrid configuration. Besides of the subject of “mail migration”, Hybrid configuration defines a “relationship” between the Exchange on-Premises server and Exchange Online that is used for additional services such as: Mail flow, Free\Busy time and much more.

Migration batch The mail migration is implemented by creating a new migration batch form the Exchange Online web management console. The migration batch is a logical concept that serves for holding information about:

The names of the mailbox that we want to migrate

The information about the “end point”

A location for saving and presenting mail migration logs and information about the mail migration process

One exception for the concept of “information about the names of the mailboxes that we need to migrate” is when we use the option of Cutover migration. This type of mail migration is based on the basic assumption that we need\want to migrate 100% of the mailboxes.

End Point The “initialization” of the migration process, is implemented by creating a logical connector from the Exchange Online that described as “End Point”. The end point is just a configuration file that includes the following details:

The required credentials (administrative user name and password) that will be used by the Exchange Online when he connects to the “mail server” (Exchange on-Premises server or other mail server).

The name of the domain and the mail server name. (In case that the server is Exchange on-Premises server, the domain name will be used in the AutoDiscover process).

The value of the maximum concurrent sessions (the maximum number of concurrent mailbox migration).

In the following screenshot we can see the setting of the “End point”

Mail migration | Copy verses Move

An additional classification that we can define, is related to the process of copy mailbox data to Exchange Online versus the process of “move mailbox data” to Exchange Online. Cutover migration and stage migration methods will only copy the mailbox data to Exchange Online versus the Hybrid migration method, that moves the Exchange on-Premises server mailbox content to the cloud (after a successful migration process the mailbox will be deleted from the Exchange on-Premises server database).

Hybrid migration verses Cutover and Stage mail migration | total amount of data

In this section, I would like to highlight a very important issue that relates to the “amount of the data” that is involved in the mail migration process, when using Native Office 365 mail migration tools.

Most of the time, when we talk about the issue of mail migration to the cloud, we relate to the “amount of data” that we need to “transfer” from point A (the organization’s network most of the time) to point B (Office 365\Exchange Online).

A very important detail that is neglected most of the time, is the amount of data that we “sent back” from the cloud – to the organization’s network (and will affect the network bandwidth of the organizational communication lines). When talking about the Native Office 365 mail migration tools, we can classify the mail migration options to two groups: Cutover, Stage and IMAP migration versus Hybrid migration.

Cutover, Stage and IMAP migration When we perform mail migration using one of the following migration options: IMAP, Cutover and Stage migration, we need to calculate the amount of data that will traverse the organization communication line by using the following formula: Mailbox size X 2

The reason for this formula is that, in the first time, we need to migrate the existing mailbox content from the organization to the cloud.

In the following diagram, we see an illustration of scenario in which we migrate 15GB mailbox to the cloud (Exchange Online).

The detail that is not mentioned most of the time, is that when using IMAP, Cutover and Stage migration, the existing Outlook OST file is no longer valid.

After the migration process of a mailbox will successfully be completed, we will need to create a new Outlook mail profile. Because the Outlook client is configured to use the option of cache mode, that will lead to a creation of a new OST file (the “old OST” file will just reside in the user profile folder, and there is no way to use this file), and Outlook client will automatically start to “pull” the mailbox content from the Exchange Online mailbox to the local user desktop OST file.

In our scenario, the Outlook client will need to “pull back” the 15GB of data to the local OST file.

Hybrid migration When using the option of Hybrid migration, we are avoiding from need to double the size of the mailbox data because, when using the option Hybrid migration, we use the term “Move mailbox” (instead the term of “Copy mailbox” when using IMAP, Cutover and Stage migration).

The mailbox content is “moved” from Exchange on-Premises server to the cloud and the Outlook client “know” how to use the existing OST file + update the existing Outlook mail profile that will point to the “new mail server” (to the name of the Exchange Online server. If we want to be more accurate, the name of the Exchange Online CAS server).

2. PST migration

Additional type of migration could be described as: “PST Migration”. I relate to this type of migration as “client side mail migration” because, the concept of PST migration is implemented by exporting the mailbox data to a PST file, create an Exchange Online mailbox and import the data from the PST file, into the Exchange Online mailbox.

I cannot refrain from expressing my opinion about PST migration. My recommendation is to try to avoid from this type of migration because, I think that this migration type is not effective and include a couple of major disadvantage’s verse the more optimal methods such as using the Office 365 built-in mail migration tools.

When using the option of PST migration, the data is “push” to Exchange Online mailbox from the user desktop. Its truth that PST Migration doesn’t need to double the size of the data by copy the data to the cloud and copy again the data form the Exchange Online to the user desktop (the method that is used when using Cutover and Stage migration) but, the process of exporting the data from Outlook and importing back the data to the user desktop + synchronizing the data to the Exchange Online mailbox is quite tedious, prevent from the user to access his mailbox as long is the export\import process is implemented and complicate the mail migration process because the person that is implementing the mail migration, will need to access separately each of the user desktops for creating the mail migration procedure.

Note – there is an option for implementing PST migration in a more “centralized way” by using a free Microsoft toll named: PST capture. You can download the PST capture tool form the following link: Microsoft Exchange PST Capture 2.0

3. 3rd Party Migration

I will not go into details regarding the subject of: 3rd Party Migration tolls\software because of the simple reason that I don’t have a lot of experience with 3rd Party mail migration tolls. Generally speaking, 3rd Party Migration tools can implement the mail migration process to Exchange Online, by using the following methods:

MAPI Migration (for my opinion, a More suitable term is RPC/HTTPS, but this is a term that mentioned in the Microsoft article).

EWS Migration

PST migration

Organization mail server technology

Microsoft Native Office 365 mail migration tools, was designed to provide a solution that is based on the basic assumption that – the organization’s mail server is a “Microsoft Exchange server.”

Note – The exception for this concept is, the IMAP mail migration method, which enables us to implement mail migration form mail server the support the IMAP4 protocol and doesn’t have to be a “Microsoft Exchange server.

In the real life, there are many other mail servers such as: IBM lotus notes, Novell GroupWise and more. In these scenarios, most of the time, the only option is to purchase 3rd Party Migration product and, usually, 3rd Party mail Migration products, has additional enhancement and features that isn’t included in the Native Office 365 mail migration tools.

Scenario 1: 3rd Party mail Migration products using an “external component” in this scenario the mail migration is implemented by using an “external component” that has two logical connectors. One connector is connected to the organization’s mail server (Microsoft Exchange on-Premises server or other mail server) and the other “leg” is connected to the Exchange Online. The 3rd Party mail migration component is responsible for “pulling” the data from the organization’s mail server and “transfer” the data (mailbox content) to Exchange Online.

Scenario 2: “Migration server“ The second methodology that is implemented by 3rd Party mail migration products described (by Microsoft) as a: “Migration server” or “jump box”. The Migration server becomes the “hub” of the migration process.

The mail server mailbox data is “transferred” to the Migration server who “hold and manage” the data and use a connector to the Exchange Online that will use for “transferring” the data to the cloud.

Note – regarding the issue of “double the mailbox data” that is caused by the need to copy the mailbox content from the organization mail server to the Exchange Online mailbox and back, when Outlook will “Pull back” the data from the Exchange Online mailbox to the local OST, it looks to me that most of the 3rd Party mail migration products will not have the ability to use the local Outlook OST file (the method that is implemented by using the Hybrid migration) but the best practice is to ask to the software providers about this option.

Exchange and the Migration engine

When we are relating to mail migration from Exchange server, there are a couple of methods that we can use for communication with the Exchange server and asking to “pull out” the data (user’s mailbox content). The available options are:

RPC/HTTPS

HTTPS/EWS

IMAP

Note – I will not relate to the IMAP migration method basic assumption is that this method will be used for mail migration from non-Exchange servers.

RPC\HTTPS and EWS

The “communication channel” that is created between the Exchange Online sever and the Exchange On-Premise is implemented by using the HTTPS protocol.

To be able to simplify the concept of RPC\HTTPS and EWS we can relate to this method as “Exchange listeners” the way the Exchange server “listen”, provide services and communicate with other hosts such as: Mail client (Outlook AnyWhere) or other servers  (in our scenario Exchange Online).

The Exchange EWS (Exchange Web services) method is more advanced and effective, and when relating to the subject of mail migration from Exchange, this is the preferred (if passable) method.

Exchange Web Service (EWS) is the recommended protocol to use for migrating to Office 365 because it supports large data batches and has better service-oriented throttling. In Office 365, when used in impersonation mode, migrations using EWS don’t consume the user’s budgeted amount of Office 365 EWS resources, consuming instead a copy of the budgeted resources:All EWS impersonating calls made by the same administrator account are calculated separately from the budget applied to this administrator account. For each impersonation session, a shadow copy of the actual user’s budget is created. All migrations for this particular session will consume this shadow copy. Throttling under impersonation is isolated to each user migration session.

[Source of information: Exchange Online Migration Performance and Best Practices ]

In the following chart, we can see the three Exchange “listeners” that we can use for “puling” the mailbox data from the Exchange server.

Microsoft Native Office 365 mail migration tools and the communication with Exchange on-Premises server

When we use the Microsoft Native Office 365 mail migration tools for “pulling” data from Exchange on-Premises server, the communication with the Exchange on-Premises server “listeners” will be implemented as follows: Cutover and Stage migration, will communicate with the Exchange on-Premises server using RPC, or if we want to be accurate: RPC/HTTPS (the real layer structure is actually MAPI/RPC/HTTPS).

Hybrid migration considers as more sophisticated because, the communication with the Exchange on-Premises server is implemented by addressing the “EWS listener” and by using a built-in component named MRS proxy, who is responsible for handling and managing the move mailbox requests (from the Exchange on-Premises server\s to the Exchange Online in our scenario).

When the mail migration is implemented by using the Exchange EWS, the result of the mail migration throughput is better than the “other” methods such as RPC and IMAP because the Exchange EWS was designed to handle a large amount of data and multiple connection in an optimal way.

In the following diagram, we can see the how different mail migration option is the Exchange “listeners” for implementing mail migration. When using a Third party mail migration product, it’s recommended to ask the provider about the specific method (RPC, EWS and so on) that the product use.

A quick reference for the article series

Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 The second article includes a review the different factors that impact the performance of the mail migration throughput. Mail migration to Office 365| Optimizing the Mail Migration throughput | Part 3/4 The third article recommendation and best practices for improving and optimize the performance of the mail migration throughput. Mail migration to Office 365 | Measure and estimate Mail Migration throughputs | Part 4/4 The last article in this series is dedicated to information about the “expected mail migration throughput” and includes a nice Excel based utility (Office 365 – Multiple and Single Mailbox Migration throughput calculator) that will help us to provide an estimation for the expected data transfer rate. Based on this information we can provide a reasonable “end date” for the completion of the mail migration project.

Additional reading

Exchange RPC and EWS Protocol Test Suites

Error message when you try to migrate users from an on-premises Exchange Server environment to Exchange Online in Office 365: “Unable to connect to the server

How to migrate mailbox data by using the Exchange Admin Center in Office 365

Migrate Mailboxes to the Cloud with a Staged Exchange Migration

Now it’s Your Turn! We really want to know what you think about the article

[o365info-blog]

New Post has been published on o365info.com

New Post has been published on http://o365info.com/mail-migration-office-365-optimizing-mail-migration-throughput-part-34/

Mail migration to Office 365 | Optimizing the Mail Migration throughput | Part 3/4

In this article, we review the recommendation and “best practices” that can help us to improve and optimize the results of the “moving our mail infrastructure to the cloud” project. Generally speaking, there are two factors that we can measure:

 The mail migration throughput – the time it takes to transfer a specific amount of data to the cloud (Exchange Online).

The amount of time required for completion of the mail migration. For example, in case that the management instructs us to migrate of all the organization mail infrastructure to the cloud in a period of no more than 10 days, can we accomplish this task?

In the former article (Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 ) we have to review the factors that impacting the performance and the result’s (data transfer rate) of the mail migration.

In this article, we review some of these factors and the options that are available to us for improving and optimize the results of the mail migration throughput or the time that we will need to accomplish the mail migration project.

A quick reference for the article series

Office 365 Mail migration | Migration methods | Part 1/4 The first article on this series includes a description of the different mail migration options that are available for us for migrating existing mail infrastructure to Office 365 (Exchange Online) and focus on the features and the characters of the different mail migration methods that relate to the mail migration. Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 The second article includes a review the different factors that impact the performance of the mail migration throughput. Mail migration to Office 365 | Measure and estimate Mail Migration throughputs | Part 4/4 The last article in this series is dedicated to information about the “expected mail migration throughput” and includes a nice Excel based utility (Office 365 – Multiple and Single Mailbox Migration throughput calculator) that will help us to provide an estimation for the expected data transfer rate. Based on this information we can provide a reasonable “end date” for the completion of the mail migration project.

Factors that impact mail migration throughput

In the following section, we will review some of the factors that impact the throughputs (transfer rate) of the mail migration and the options that are available for use to fine tune or optimize these factors to improve the results of the mail migration throughputs.

Network infrastructure

1. Communication line bandwidth

1.1 The “speed” (The bandwidth) of the organizational communication line To be able to measure the performance of our communication line, we can use a web tool that is provided by the ISP or use public tools such as: speedtest

Using the results we can get better understanding about the available resources that we can use such as: the Upload rate (when we are moving an organization mailbox to the cloud, we use the “upload channel), the download rate and son on.

Based on the results that we get, we can decide if there is a need to Increase the bandwidth of the existing communication line.

In case that we want to get more detailed information about the load and performance of the communication line during the day, week, etc. Most of the ISP can provide this information (for free most of the time), by accessing a web interface that will display daily, weekly and monthly results of the communication line performance. Personally, I like to use a product named: PRTG  that enables us to get a very clear and graphic view of the network performance.

1.2 working hours and weekends Office 365 native mail migration tools enable us to schedule the mail migration process. In case we want to avoid from a scenario, in which the mail migration will “load” the organization communication line (and downgrade the network performance -the communication lines available bandwidth) or in case that we want to use all the available communication line bandwidth, we can consider scheduling the mail migration process to “non-working hours” or implementing the mail migration on the weekends.

1.3. Network devices performance and tuning

Firewall or Proxy load and performance To be able to verify that the existing network device such as Firewall or Proxy, are not the cause for the “Bottle neck” or slowing the performance of the mail migration, the best practice is to implement a Pilot of mailbox migration and monitor the load performance of the Firewall\Proxy to verify that the server utilization is not high or causing the Firewall\Proxy to “reach their end limit.

Another important factor that can impact the performance of the mail migration is configuration settings that restrict the maximum value of the “allowed sessions.”

Firewall\Proxy | passable restriction of the maximum allowed sessions Check the Firewall\Proxy server to verify that the migration process session is not blocked or restricted by the Firewall\Proxy policy.

For example, you can read the following article that describes and issue that exists when using TMG server and performing a mail migration: Office 365 Move Mailbox fails with transient exception 1.4. Geographic location of the organization’s network One of the factors that affects the mail migration throughput to Exchange Online, is the “physical distance” between the organization network and the Office 365 data centers.

Technically, is not easy to measure or assess the quality of the of the communication path from point A (organization network) to the destination (Office 365 data center) but, the good news is that Microsoft provides a nice Web-based tool named: Office 365 Network Analysis Tool, that can help us to get a detailed information about the performance of the communication channel between a specific organization network and the Office 365 data center that hosts the organization mailbox.

The Office 365 Network Analysis Tool is a Java based tool and implemented as a web application. To be able to get the required results we will need to choose the suitable URL for the Office 365 data center that hosts the organization mailbox, specify our Public domain name that is registered at Office 365 and the tools will start to collect information and on the next step, display detailed results.

North America: http://na1-fasttrack.cloudapp.net

EMEA: http://em1-fasttrack.cloudapp.net

APAC: http://ap1-fasttrack.cloudapp.net

To be able to use the Office 365 Network Analysis Tool, we will first need to download and install the java client. In case that try to use the tool and get the following error “application blocked by a security setting“, go to the control panel, look for the icon named: Java, and in the security tab change the default settings to: medium

In the first screen we will need to provide the domain name that is hosted in Office 365.

When the scanning process is completed, we can browse by selecting the different tabs to get a detailed information about a specific section of the Findings

When choosing the speed tab we can see a graph that include information about the download and the upload speed.

When choosing the capacity tab we can get more details about the download and upload performance.

1.5. Using a QOS (quality of service) product Most of the time the important question is not: “what the bandwidth of the communication line” but instead: “what is the utilization of the communication line or what is the “free percentage’” of the communication line.

The process of mail migration considers as demanding because, most of the time we “transfer” a large amount of data from the organization’s network to the cloud.

In case that your organization uses QOS product, the best practice is to allocate a predefined percentage of the network bandwidth for the task of the mail migration or prioritize communication that relates to the mail migration.

1.6. Plan for the required communication line bandwidth When relating to the concept of “Plan for the required communication line bandwidth”, one major phase is the bandwidth that is required for the mail migration (the mailbox data that will be transferred from the Exchange on-Premises server to the cloud), and additionally, we will need to plan ahead for the required communication line bandwidth that will be allocated to the communication between the organization users who access there Exchange Online mailboxes.

To be able to estimate the required network bandwidth is recommended to use tools such as: Exchange Client Network Bandwidth Calculator

The Exchange Client Network Bandwidth Calculator is based on excel shit. On the first screen, we need to provide some general information about our mail infrastructure.

The second tab will help us to display a clear picture of the requirements that reacted for the Network bandwidth for each of the different types of mail clients.

Attached relevant links

Exchange Client Network Bandwidth Calculator

Exchange Client Network Bandwidth Calculator v2

Plan for Internet bandwidth usage for Office 365

Exchange On-Premise infrastructure

1. Data source (Exchange Server) – Performance The migration process is going to put a heavy load on the Exchange on-Premises servers who host the user’s mailboxes or in case that we use hybrid configuration of the Exchange on-Premises hybrid server.

Before we start the migration process, the best practice is to ensure that the existing Exchange on-Premises servers have the require resources (RAM, CPU, Storage, etc.) that will enable this server to perform in an optimal way.

Q: how can I know if the migration process is overwhelming the Exchange on-Premises server? A: Start low as 10 and then increase this number while monitoring the data source performance to avoid end-user access issues.

In my opinion, the option of “upgrading existing Exchange on-Premises server hardware” is most relevant in case that the Exchange on-Premises server is based on a virtual machine because, in this scenario, the hardware upgrade process is a “one-click click procedure.”

Hybrid migration | Server performance

In case that you use a Hybrid server that will use as a “Router” for the mail migration process, the best practice is to use powerful server-class physical machines instead of virtual machines for the Exchange 2013 and 2010 hybrid servers. You can read additional information by using the following links:

Server Health and Performance

Understanding Exchange Performance

Monitoring Mailbox Servers

2. Tuning the MRSProxy resource allocation

The MRSProxy is the Exchange server component that is responsible for implementing a mailbox move (in our scenario mailbox move from Exchange on-Premises server to Exchange Online).

The tasks that are executed by the MRSProxy described as: connections. The default value for the maximum number of connections is 100 but, we have the ability to “tune” these values based on our needs.

Scenario 1: Exchange server overwhelmed by the migration load in case that we find that the migration process impairs the performance of the Exchange on-Premises server, we can reduce the maximum number of connections that the MRSProxy will be able to use. Scenario 2: in case that the existing Exchange on-Premises server has sufficient hardware resources, and we want to expand more resources to the MRSProxy, we can increase the default number of connections that will be available for the MRSProxy and by doing so, optimizer that mailbox migration transfer rate.

The PowerShell command that we can use is:

Set-WebServicesVirtualDirectory -Identity "EWS (Default Web Site)" -MRSMaxConnections <number between 0 and unlimited; default is 100>

3. Exchange on-Premises server throttling policy

In case that you have an implementation of Exchange on-Premises server throttling policy, that could slow or “downgrade” the performance of the mail migration process, consider stopping or disabling these settings while the mail migration is running.

You can read additional information in the following article: Throttling Policy Associations in Exchange 2010 SP1

Using Hybrid migration if passable Using the option of Hybrid migration decreases the amount of data related to mail migration.

In the former article ( Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 ) we describe the difference that exists between Hybrid migration verses stage migration and Cutover migration regarding the amount of data that is involved in the mail migration process.

Just a short recap: when using the option of Hybrid migration, we don’t have to configure a new Outlook mail profile but instead, we can use the same Outlook mail profile and the existing OST file. By doing so, we are avoiding from the need to “download back” of the mailbox content to the user desktop.

Note – I’m not familiar with each of the third party mail migration tools that exists and if there are third party mail migration tools that “know,” how to use the local Outlook OST instead of downloading all the mailbox data at a new OST file.

Using the option of: Hybrid migration minimum is based on the minimum requirements of Exchange 2010 SP3 on-Premises server. In a scenario in which the organization mail infrastructure is based on older Exchange on-Premises server such as: 2003, 2007, I would suggest you consider the option of installing a “temporary Exchange 2010 on-Premises server” for the mail migration project.

The Interesting fact is that you don’t even need to pay for a license. Microsoft enables an organization that wants to install Exchange 2010 on-Premises server only as a hybrid server (without using the mailbox rule option) to get a free license for this server.

Attached a link which leads you to the required license enrollment process: Self-service Hybrid Edition product key retrieval for customers

Exchange multiple site infrastructure

Mid and large organization will usually have more than one Exchange site. In case that your organization uses more than one Exchange site (Multiple Exchange sites) and in case that each of these Exchange sites has a “public presence” that enables to external hosts to connect the Exchange on-Premises server who “represent” the Exchange site, we can significantly improve the mail migration throughputs.

When relating to the subject of mail migration to Office 365 and multiple mailbox migrations (concurrent mailbox moves), we can use the famous saying “the more the merrier”

Note – is assumption is correct provided that the mail\communication line infrastructure can provide the required resources.

For example, in our scenario, we need to migrate 300 hundred mailboxes from the organization Exchange on-Premises servers to the cloud. We have a very narrow time window for completing this task and, we would like to shorten or to expedite to the process of the mail migration.

In the following diagram, we can see that by using the concept of implementing a mail migration from a multiple Exchange site at the same time (parallel), we can multiply or triangles the amount of mailboxes that we can migrate at the same time.

Multiple Exchange site’s infrastructure and AutoDiscover

In a scenario of Multiple Exchange site’s infrastructure, we can rely on the AutoDiscover service for implementing a mail migration from a multiple Exchange sites at the same time, based on the location of the user mailbox (the Exchange site and the Exchange server who hosts the user mailbox) The pre-requirements is that each of the Exchange sites will have a public presence that includes “Public name” (URL) that can be accessed from a public network.

To be able to understand better this process let’s use the following example: An organization named: o365info have two Exchange sites. One site is located in New York, and additional site is located in Los Angles.

 The Exchange mail server in the New York site uses the following public name: Mail.Ny-o365info.com

 The Exchange mail server in the Los Angles site uses the following public name: Mail.La-o365info.com

We are using the option of Hybrid migration, and we are creating a new migration batch that we be based on a CSV file that includes a list of users whom we want to migrate to the cloud (Exchange Online). The CSV users list includes a “mixture” of users from booth of the organization site.

In our example: User1 and User2 mailboxes are located in the New York site and User3, and User4 mailboxes are located in the Los Angeles site.

In our scenario, the AutoDiscover record of o365info.com, is pointing to the additional Exchange server who is not located on the company site. When we start the migration batch, Exchange Online uses the AutoDiscover service for getting the required information about the mail infrastructure etc.

In the following diagram, we can see that Exchange Online ask about the Exchange server (The CAS server) that can provide information and access to the mailbox data of [email protected] The origination Exchange server provides an “answer” by redirecting the Exchange Online to additional server: Mail.Ny-o365info.com In the second step, Exchange Online will try to connect to the Mail.Ny-o365info.com and “ask for a permission” to copy the mailbox data of User1.

Exchange Online creates a new migration endpoint using the connection settings that were successfully discovered or that you provided manually. We recommend that you create migration endpoints whose connection settings were automatically discovered rather than creating endpoints whose settings you entered manually. This is because the Autodiscover service will be used to connect to each user mailbox during the migration. If manual settings are used, Exchange Online won’t use the Autodiscover service, but will connect to a specific source server using the connection settings you manually entered. If you use manual settings and have multiple on-premises Exchange servers, you may need to create different migration endpoints that correspond to each server.

[Data source: Exchange Online Migration Performance and Best Practices ]

Additional reading

Create Migration Endpoints

New-MigrationBatch

Create Migration Endpoints

How to migrate mailbox data by using the Exchange Admin Center in Office 365

  Office 365\Exchange Online factors

1. Office 365\Exchange Online Throttling policy

The Throttling policy that relates to mail migration described as: Migration-service throttling. This policy was designed to limit the maximum migration concurrency.

The good news is that we can easily increase the default value and by doing so, Increase the number of mailboxes that can be migrated.

In my option, the default value of: “maximum 20 mailboxes at the same time” was not created to protect the Exchange Online server performance because theoretically, there is no limitation to the resources that allocated to the Exchange Online server. The attention was to protect the Exchange on-Premises server from a scenario of “overwhelming”. In case that we put too much load on the Exchange on-Premises server, the load of the migration process, could impact other Exchange on-Premises server services.

So, in case that you are considered to increase the value of the Migration-service throttling, please verify that your Exchange on-Premises server, have the required resources for supporting the amount of mailboxes that will be migrated to Exchange Online.

After you create a migration batch, you can use the following Windows PowerShell command to increase this to a maximum of 100.

Set-MigrationEndPoint <Identity> -MaxConcurrentMigrations <value between 1 and 100>

2. The maximum number of concurrent moves for a Migration Endpoints

The information about the subject of: The maximum number of concurrent moves for a Migration Endpoints is a little confusing. A public Microsoft article that was published on 2011-12-02 provides the following information:

However, after you create a migration batch but before you start it, you can use Windows PowerShell to increase this number to a maximum of 50 mailboxes to migrate at the same time.”

[Data source:Increase the Number of Mailboxes to Migrate Simultaneously ]

The more updated Microsoft article includes information about different value

There is a limit of 100 concurrent migrations for each type of migration endpoint. For example, your organization could have five Outlook Anywhere migration endpoints and each endpoint could be configured for 20 maximum concurrent migrations. Or you could have two IMAP migration endpoints, with each end point configured to support 50 maximum concurrent migrations.

[Data source:Create Migration Endpoints ]

In the following screenshot you can see that when we try to use a value bigger than 100

The result is the following error:

Just a friendly advice: the fact that Office 365 enables us to “extend” the value of a concurrent mailbox move doesn’t mean that you should automatically use this option.

A high value of concurrent mailbox moves can overload your network bandwidth and you Exchange on-Premises servers.

The Number of mailboxes to migrate simultaneously field specifies the number of connections to your on-premises mail server used during migration. The more connections you have to your on-premises mail server, the more mailboxes you can migrate at one time. While this reduces the amount of time it takes to migrate mailboxes, these additional connections can consume your Internet bandwidth, negatively affecting network performance and users’ ability to access Internet resources.

[Data source : Increase the Number of Mailboxes to Migrate Simultaneously ]

A quick reference for the article series

Office 365 Mail migration | Migration methods | Part 1/4 The first article on this series includes a description of the different mail migration options that are available for us for migrating existing mail infrastructure to Office 365 (Exchange Online) and focus on the features and the characters of the different mail migration methods that relate to the mail migration. Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 The second article includes a review the different factors that impact the performance of the mail migration throughput. Mail migration to Office 365 | Measure and estimate Mail Migration throughputs | Part 4/4 The last article in this series is dedicated to information about the “expected mail migration throughput” and includes a nice Excel based utility (Office 365 – Multiple and Single Mailbox Migration throughput calculator) that will help us to provide an estimation for the expected data transfer rate. Based on this information we can provide a reasonable “end date” for the completion of the mail migration project.

Additional reading

Maximum Number of Connections to Your Mail Server

can I create more then one migration end point to speed up the process for cutover migration

Increase the Number of Mailboxes to Migrate Simultaneously

Now it’s Your Turn! We really want to know what you think about the article

[o365info-blog]

New Post has been published on o365info.com

New Post has been published on http://o365info.com/mail-migration-office-365-factors-impact-mail-migration-throughput-part-24/

Mail migration to Office 365 | Factors that impact mail Migration performance | Part 2/4

When we implement a project of mail migration to the cloud (Exchange Online), one of the most impotent measurement values that we use is the “throughput value” of the mail migration. In simple words, we want and need to know, how much time it will take us to copy data (mailbox content) from the organization to the “cloud” (Exchange Online).

This “value” is important to us because based on this value, we can provide an estimation for How long it will take to finish the project, what are the costs that involved in the project (a longer project will cost more than shorter time project) and so on

However, before we can provide the required estimation, we need to know what are the “entity” or the factors that are involved in the mail migration process and how this factors impact the mail migration process.

Note – In the next article (Office 365 mail migration | Optimizing the throughput | Part 3/4 ) of this series, we will review these factors and the options that are available for use for improving and optimize these factors.

A quick reference for the article series

Office 365 Mail migration | Migration methods | Part 1/4 The first article on this series includes a description of the different mail migration options that are available for us for migrating existing mail infrastructure to Office 365 (Exchange Online) and focus on the features and the characters of the different mail migration methods that relate to the mail migration. Mail migration to Office 365| Optimizing the Mail Migration throughput | Part 3/4 The third article recommendation and best practices for improving and optimize the performance of the mail migration throughput. Mail migration to Office 365 | Measure and estimate Mail Migration throughputs | Part 4/4 The last article in this series is dedicated to information about the “expected mail migration throughput” and includes a nice Excel based utility (Office 365 – Multiple and Single Mailbox Migration throughput calculator) that will help us to provide an estimation for the expected data transfer rate. Based on this information we can provide a reasonable “end date” for the completion of the mail migration project.

General mail migration questions

Before we begin, I would like to ask would like to ask two questions (that will be answered in the current article and in the next articles in this series). Q1: Is there any kind of a formal answer about a “fixed number” that we can use, when we need to estimate the throughput of mail migration (the transfer rate of data)? A1: Yes, it is true that there is not much information on the subject but, there is a Microsoft official article that provides “numbers” and estimation for a mail migration throughput.

Q2: Why do we need to use an “estimation” or “average range of values” for the mail migration throughput? Why we cannot use a simple a mathematical formula for calculating the required time for transfer information from point A to point B? A2: The answer is that – when we migrate a mailbox from point A to point B, there are many components that are involved in the process. Each of these “components” have a specific character that impact the result of the mail migration throughput.

When we want to migrate the content of a user’s mailbox to the cloud (Point B), we need to get through many elements. Each of these “elements” have the ability to “hold” the process or just let the data get through.

Each of the “elements” or the factor have resource’s limitations that affect the performance. For example, in case that the “element” is the local Exchange On-Premise server, the mail migration throughput is depended or based on the Exchange On-Premise server resources such as: CPU, RAM, Hard disk and so on.

Additional factors are the type of the mail migration that we use, limitation of the communications line, Throttling policy, the size of the mailbox, the number of mail items and much more.

Factors that impact mail migration throughput

In the following section, we will review some of the factors that impact the throughputs (transfer rate) of the mail migration. Additional and important information about the performance factors that is not included in the current articles, appears in the article: Exchange Online Migration Performance and Best Practices

Network infrastructure

1. Communication line

One of the most important factors that relates directly to the result of the mail migration transfer rate is obvious, the existing communication line.

1.1 The “speed” (The bandwidth) of the organizational communication line Regarding the rate or the speed of the communication line, don’t forget that many times the communication line technology is providing different transfer rate for “upload’” verses “download” (asynchronies). When we migrate existing mail infrastructure to the cloud, the operation is considered as “upload” (moving data from the local network to the public network) and many times, the transfer rate for “upload” is slower or less than the download rate.

1.2 The free or available percentage of the organization communication line A very important factor is the “load” that exists on the communication line or the “free percentage bandwidth” of the communication line. For example, it’s not enough to say that the existing communication line bandwidth is 20MB, but instead, the real question is how much percentages of the bandwidth are free or available for the task of the mail migration project.

1.3 Working hours and weekends The additional element that affects the communication line “free or available bandwidth” is – the factor of working hours and weekends. The basic assumption is that during the “working hours,” the amount of the “free or available bandwidth” of the communication line, is lower than the rate that we can get after the working hours and especially at weekends.

2. Geographic location of the organization’s network

Office 365 data centers and mail infrastructure are physically located in different geographical location worldwide. There are three “main point” in which Office 365 Data centers are located. Here is a quotation from an article that I have found:

The way I’ve always answered this question is that Microsoft divides the world into three regions: North & South America, Europe & Africa, and Asia & Australia. We have an undisclosed number of datacenters in each of those regions, and we select a primary datacenter within the region where the organization’s headquarters address is located

[Source of information: Where is my Office 365 data located? ]

Additional important question that we can ask is: in case that our organization has multiple sites that are located worldwide, can we assume that each location is configured to access the nearest Office 365 data center?

The answer is: “No”. All the organizational resources such as user’s mailboxes will be located in one data center and will not be “distributed” between the various Office 365 data centers worldwide.

Microsoft has a regionalized data center strategy. The customer’s country or region, which the customer’s administrator inputs during the initial setup of the services, determines the primary storage location for that customer’s data.

For specific details about where are the Office 365 data center geographically located, you can read more information in the article: Where is my data?

The point of this information is that- there is an impact for the geographic “distance” between the organization’s network and the Exchange Online data center. (I will provide more information about how to check the quality of the communication and provide a recommendation in the last article of this serious Office 365 mail migration | Optimizing the throughput | Part 3/4 )

3. Network devices

Network devices such as a Firewall or Proxy servers can impact the results of the throughput (data transfer rate) in a couple of ways:

The load factor- The load or the performance of these devices: for example, in case that the existing Firewall or Proxy server has very high utilization, this could lead to a reduction of the mail migration throughput.

Number of session restrictions- Another factor could be: the feature of Firewall and especially Proxy server that limits or restricts the number of sessions in the communication process.

For example: the process of mail migration could create tens or hundreds of sessions, between the Exchange on-Premises server and Exchange Online. In case that the Firewall or the Proxy server is configured with a maximum number of session’s value, and the mail migration process reaches this limit, the throughput of the mail migration is reduced.

Exchange On-Premise infrastructure

1. Data source (Exchange Server) – Performance of the local Exchange On-Premise servers.

Mail migration is based on the concept in which the mailbox data is copied or moved to the cloud (Exchange Online). In a scenario of multiple mailboxes, the amount of the data that is transferred can easily become huge and the operation of accessing to each of the mail server mailboxes and copy the data creates a considerable load on the Exchange on-Premises server.

In a scenario of Hybrid configuration, the server load could impact the Exchange on-Premises server mailbox server who hosts the user’s mailboxes or to the Exchange on-Premises server hybrid server, which serve as a router between the internal Exchange on-Premises servers and the cloud.

2. Exchange sites

Mid and large organization, will usually have more than one Exchange site. The subject of Exchange sites can impact the data transfer rate of the mail migration from two perspectives:

The geographic location of the Exchange site versus the geographic location of the Exchange Online data center. In case that the Exchange site is “closer” to the Exchange Online data center, we would probably have better results.

Performing mail migration by using multiple Exchange sites In case that the mail migration project is implemented by performing the mail migration from multiple Exchange sites, we can migrate more mailboxes in a specific time window and the result of the mail migration throughput is significantly improved.

3. Hybrid migration & MRSProxy

When using the option of Hybrid migration, the Exchange on-Premises server component that is responsible for – implementing and managing the move mailbox operations is the MRSProxy. The task of moving mailbox from source server to the destination server described as: MRSProxy connections. In Exchange on-Premises server, the default value for the MRSProxy for the maximum number of connections is 100. In case that we want to increase this value, we can change the default value by using a PowerShell command (we will review the exact syntax in the Office 365 mail migration | Optimizing the throughput | Part 3/4 )

4. Exchange Mailbox charters

Most of the time we mention the term “mailbox,” we use this term in general. In the reality, the term “mailbox” is used for describing many different types of mailboxes such as:

Small-sized mailbox with a small amount of mail items

Average mailbox with the average number of mail items

Small-sized mail box with a large number of mail items

Large-sized mailbox that usually characterizes as a mailbox with large-size + large amount of mail items.

So what is my point?

My point is – that most of the time we relate only to the “mailbox size” and we understand that logically, it will take more time to migrate Large-sized mailbox versus a Small-sized mailbox. An important factor that is not mentioned most of the time that has a tremendous impact on the mail migration throughput is: the number of mail items that exists in the user mailbox.

Common migration performance factors One 4GB mailbox with 400 items, each with 10 megabytes (MB) of attachments, will migrate faster than one 4-GB mailbox with 100,000 smaller items.

[Data source: Exchange Online Migration Performance and Best Practices ]

  Number of mail items and third party mail migration tools In the article:  Exchange Online Migration Performance and Best Practices, We can find a data table that focuses on the mail migration throughput results in a scenario on a mailbox with an average amount of mail items versus a mailbox with a lot or above average of mail items when using an RPC mail migration.

The little “catch” is that the data table relates to a result of mail migration that was created when using a third party mail migration tools.

(I must admit that I’m wondering why the article includes information about third party mail migration tools and doesn’t include a related information to the Office 365 built-in migration options)

Although the table compares the migration throughput in a scenario of third party mail migration products think that we can use the information as a derivative for all of Office 365 mail migration options such as: Cutover and Stage migration because, these methods are also based on an RPC connection that is created between the Exchange on-Premises server and the Exchange Online server.

So what is the data telling us? If we take a closer look at the data that appear in the table, we can observe very inserting issue: when we migrate an Exchange mailbox (small size mailbox in our scenario) that have multiple numbers of mail items the process will create a huge amount of RPC transactions.

To emphasize the conclusion form the data that appear in the table, I have created the following diagram.

In this scenario, we can see information about the mail migration throughput for two mailboxes. One mailbox is quite small (size of 249 MB), and the other mailbox is larger (377 MB). The interesting fact is that – the time that we need to move the smaller mailbox is significantly longer (11~ hours for the smaller mailbox verse 4.5~ hours for the larger mailbox).

The answer for this “strange result” is that the smaller mailbox, includes 13,000~ mail items versus the larger mailbox that includes 4,100~ mail items. The conclusion is that the factor of the number of mail items in a mailbox, has a huge impact of the mail migration throughput.

5. Exchange on-Premises server throttling policy

Additional factor that we can relate to is: the Exchange on-Premises throttling policy. In case that there is an implementation of: Exchange on-Premises throttling policy, the throttling policy could impact the performance of the mail migration process. You can read additional information in the following articles:

Throttling Policy Associations in Exchange 2010 SP1

Exchange Online Throttling and Limits FAQ

Office 365\Exchange Online factors

1. Throttling policy\Throttling mechanism

The term “Throttling policy” is a term that describes a similar concept as the term QOS (Quality Of Service). A Throttling policy is basically a predefined restriction or a limitation which is enforced by the administrator on a specific system.

For example: in the Exchange server environment, we can use Throttling policy for: limiting the number of concurrent PowerShell remote sessions, the maximum amount of data that can be accepted or sent and so on. The question that relates to our scenario of mail migration to Office 365 is: Does Office 365 use Throttling policy that restricts the amount of data that can be “transferred” from the on-Premises mail server to Exchange Online? And the answer is Yes and No.

In Office 365, there is an implementation of three types of throttling policy that relates to male migrations:

Office 365 migration-service throttling

Office 365 user throttling

Resource health-base throttling

Office 365 migration-service throttling

The migration-service throttling define the maximum number of concurrent mailbox move to the Exchange Online server. For example:

The default value for concurrent mailboxes “moves” when using Cutover, Stage or Hybrid migration is 20.

The default value for concurrent mailboxes “moves” when using IMAP migration is 4.

Q1: can I change this default value? A1: Yes, very easily from the GUI interface or by using a PowerShell command.

Q2: can I assume that the obvious conclusion is: if I double the default value from 20 concurrent moves to 40 concurrent moves, I will double the mail migration throughput? A1: In theory the answer is: “yes.” We use the term “In theory,” because when we increase the number of maximum mailbox move on the Exchange Online server side (and in the Exchange on-Premises server side), the results could be: overwhelming of the Exchange on-Premises server because, the task of “move mailbox” is consuming resources from the server who “deliver” the mailboxes (Exchange on-Premises server) and the server who “accept” the mailboxes (Exchange Online).

My point is that we can assume that Exchange Online has an unlimited amount of resources, but we cannot apply the same conclusion to the Exchange on-Premises server because the basic assumption is that the Exchange on-Premises server has limited hardware resources.

Office 365 user throttling

The term “user throttling” describe an Office 365 Throttling policy, which is implemented or “enforced” in two scenarios: scenario 1: when using the option of “PST Migration”Scenario 2: when using Third party MAPI Migration (client-uploading migration method. That is implemented by using RPC over HTTPS).

Q1: What is the reason for using the user throttling policy? A1: I could not find a formal answer for this question, but I can assume that the reason for implementing the user throttling policy when using PST migration (client-uploading migration) is that this type of migration, is not very effective and cause a high load on the side that “export the data” (The client side) and the side the “accept” the data (Exchange Online).

The Office 365 user throttling policy was designed for “protecting” the Exchange Online from a scenario in which we implement tens or hundreds of PST migration and by doing so, “overwhelming” the Exchange Online servers.

User throttling affects most third-party migration tools and the client-uploading migration method. These migration methods use client access protocols, such as RPC over HTTP, to migrate mailbox data to Exchange Online mailboxes. These tools are usually used to migrate data from platforms such as IBM Lotus Domino and Novell GroupWise. User throttling is the most restrictive throttling method in Office 365. Because user throttling is set up to work against an individual end user, any application-level usage will easily exceed the throttling policy and result in slower data migration.

[Data source: Exchange Online Migration Performance and Best Practices ]

Resource health-base throttling

The Resource health-base throttling was created to protect the health of the Exchange Online server in special scenarios. The use of the Resource health-base throttling will be implanted only on special occasions, when the monitoring system identifies a special load or specific problem that relates to the Exchange Online server infrastructure.

All migration methods are subject to the governance of availability throttling, but Office 365 service throttling doesn’t affect Office 365 migrations as much as the other types of throttling described in the previous sections. Resource health-based throttling is the least aggressive throttling method and occurs only when there is a service availability issue that affects end users and critical service operations.

[Data source: Exchange Online Migration Performance and Best Practices ]

Summery

The following screenshot was taken from the Microsoft article: Exchange Online Migration Performance and Best Practices

The table describes the “matrix” of the Office 365 Throttling policies and in which of the migration scenario a specific Throttling policy is implemented.

In the following diagram, we can see a different representation of the information that appears in the table: we can see that the Resource health-base throttling is implemented for all the mail migration types without exception. When “server to server migration” such as: Stage or Cutover migration, the Office 365 migration-service throttling is used to define a maximum number of concurrent mailbox move.

A quick reference for the article series

Office 365 Mail migration | Migration methods | Part 1/4 The first article on this series includes a description of the different mail migration options that are available for us for migrating existing mail infrastructure to Office 365 (Exchange Online) and focus on the features and the characters of the different mail migration methods that relate to the mail migration. Mail migration to Office 365| Optimizing the Mail Migration throughput | Part 3/4 The third article recommendation and best practices for improving and optimize the performance of the mail migration throughput. Mail migration to Office 365 | Measure and estimate Mail Migration throughputs | Part 4/4 The last article in this series is dedicated to information about the “expected mail migration throughput” and includes a nice Excel based utility (Office 365 – Multiple and Single Mailbox Migration throughput calculator) that will help us to provide an estimation for the expected data transfer rate. Based on this information we can provide a reasonable “end date” for the completion of the mail migration project.

Additional reading

Exchange Online Migration Performance and Best Practices

Exchange Online Throttling and Limits FAQ

EWS throttling in Exchange

Now it’s Your Turn! We really want to know what you think about the article

[o365info-blog]

New Post has been published on o365info.com

New Post has been published on http://o365info.com/mail-migration-office-365-measure-estimate-mail-migration-throughputs-part-44/

Mail Migration to Office 365 | Measure and estimate Mail Migration throughputs | Part 4/4

In this article, we review the subject of mail migration throughputs when migrating existing mail infrastructure to Office 365. We need the answer for the for the mail migration throughputs for a couple of reasons:

Reason 1: in a mail migration project, we need to provide a due date for the completion of the migration process.

Reason 2: when implementing the mail migration, we need to have some baseline or a reference that could help us to understand if the existing results of the mail migration throughputs are reasonable or in case that we notice that the mail migration throughputs is very low, find the reasons for the “Low throughputs” and passable solution that will help us to optimize and improve the throughputs (Transfer rate) of the mailbox content to the cloud.

A quick reference for the article series

Office 365 Mail migration | Migration methods | Part 1/4 The first article on this series includes a description of the different mail migration options that are available for us for migrating existing mail infrastructure to Office 365 (Exchange Online) and focus on the features and the characters of the different mail migration methods that relate to the mail migration. Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 The second article includes a review the different factors that impact the performance of the mail migration throughput. Mail migration to Office 365| Optimizing the Mail Migration throughput | Part 3/4 The third article recommendation and best practices for improving and optimize the performance of the mail migration throughput.

Mail migration to Office 365 common questions that need to be answered

One of the first questions that we could ask is: what is the required time for migrating a mailbox to the cloud?

The next question could be: what is the exact definition of a standard mailbox? (A user mailbox could be 1GB mailbox or 20GB with hundreds of thousand mail items).

Let’s assume that we can define the size of an “average or standard mailbox.” We still need to know, how much time it will take to migrate all the organization mailboxes to the cloud or in other words, what is the expected or the average transfer rate when migrating mailboxes to Exchange Online?

Let’s assume that we have implemented some pilot, migrate 10 mailboxes to Exchange Online, measure the average transfer rate and come off with some numeric results.

The question now is: does this result consider as bad? Average? Good? Do we have some guideline for the expected transfer rate result?

Mail migration to Office 365 | Performance of different migration methods

In the following section, we will try to provide some estimation for the expected mail migration throughput based on the Microsoft article: Exchange Online Migration Performance and Best Practices

In the following screenshot, we can see a data table named: “Performance for migration methods.”

Before we start to analyze the information on the table a couple of notes:

Many of the important details appear in the “note section” under the table.

The information in table includes some “holes” because the measurement and the results that appear in the table (and in the note section) relate to different values of concurrent mailbox’s migration. For example, the mail migration throughputs for Cutover and Stage migration relate to 100 concurrency versus the throughput’s results of Hybrid migration that relate to 20 concurrency.

To be able to display the data more clearly I have created the following table:

Migration Method Throughput Single mailbox move 20 concurrency 50 concurrency 100 concurrency IMAP Migration 10-15 GB Cutover Migration 10-15 GB Staged Migration 10-15 GB Hybrid Migration 0.3-1.0 GB 10-15 GB 15-50 GB Third party MAPI Migration 0.1-0.5 GB 4-12 GB Third party EWS Migration 0.2-0.5 GB 5-10 GB 20-50 GB Client Uploading (From Outlook PST) 0.5 GB

  General conclusions from the “Performance for migration methods”.

1. MAPI\RPC mail migration versus EWS method

As it was mentioned in the former article (Mail migration to Office 365| Optimizing the Mail Migration throughput | Part 3/4 ) ,when we perform mail migration from Exchange on-Premises server, we can “address” the Exchange on-Premises server in two ways: MAPIRPC or EWS.

By looking at the data, it is very clear that when we perform the mail migration by addressing the Exchange EWS “listeners” that mail migration throughput are much better. For example: Cutover and Stage mail migration is implemented by using MAPI/RPC connection to the Exchange on-Premises server verse’s Hybrid Migration that addresses the EWS “listener.” We can see that the results for 100 concurrency mailbox migration are very different when we compare the Cutover and Stage mail migration versus the Hybrid mail migration: the results range for Cutover and Stage mail migration for 100 concurrency is: 10-15GB and the results of Hybrid mail migration are: 20-50GB.

Note – the comparison is between the Office 365 native mail migration option versus Third party EWS Migration because the article doesn’t include information about 100 concurrency when using the option of Hybrid mail migration.

2. Single mailbox migration versus multiple mailbox migration

Another interesting information that appears in the data table relates to a scenario of: Single mailbox migration versus multiple mailbox migrations.

In the following screenshot, I have highlighted the information the estimated throughput of Single mailbox migration versus multiple mailbox migrations when we use the option of Hybrid migration. Clearly the throughput results of Single mailbox migration are inferior versus the throughput results when we perform multiple mailbox migrations.

The reason for this difference

I try to understand what the reason for this difference is. The article includes the following note:

Observed single mailbox move throughput is in the 0.3–1.0 GB/hour range. More concurrent mailbox migrations can be used to achieve higher data migration rates. For example, with 50 concurrent moves, the overall throughput will be in the 15–50 GB/hour range. Single mailbox move throughput will slow down when the on-premises CAS (MRSProxy) server is at hardware capacity. Consider adding more servers to increase migration velocity

[Data source: Exchange Online Migration Performance and Best Practices ]

  It’s not clear to me if this note relates to a scenario of performing multiple mailbox migrations from multiple Exchange on-Premises server or performing multiple mailbox migrations from a single Exchange on-Premises server, but the general conclusion is that we will get much better results when we implement multiple mailbox migrations.

3. Hybrid mail migration versus Third party EWS Migration

The preferred method for performing mail migration in Exchange based environment is addressing the EWS services of the Exchange server. The data in the table include information about EWS mail migration that is performed by using the Hybrid migration versus Third party EWS Migration. The conclusion from the results is that Hybrid migration provides a better throughputs result, then Third party EWS Migration tools.

Calculating the throughput for the mail migration

In the following section, we will try to “translate” the data in the “Performance for migration methods” table (Exchange Online Migration Performance and Best Practices ) to time units.

Range of values Let’s start with the concept of: “Range of values.” For example: the information in the “Performance for migration methods” table says that when using the option of Hybrid configuration with 20 concurrency (multiple mailbox migration) the throughput could be a value starting from 10GB per hour and ending with 20GB per hour.

The reason for using a range of values instead of proving a “single number” is that there are many possible factors (which we review in the article: Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4) that can impact the mail migration process and by doing so, provide different results.

Although it is not literally written in the Microsoft article, I assume that the “numbers” that appears in the data table are some kind of average that was created from analyzing the data (the results) from many “cloud mail migration projects.”

For simplifying the use of the data range, I have created the following diagram. In the “low range” we get a mail migration throughput of 10GB per hour, in the “High range” we get a mail migration throughput of 15GB per hour, and additionally, I have added a “Mid-range” that is collocated as the “Half Way” between the Low range and the High range. In our scenario, the “Mid-range” mail migration throughput of 12.5GB per hour.

Q1: Can we use these values\number as an absolute number when we need to provide an estimation for the mail migration throughput?

A1: My opinion is that we cannot relate to this numbers as an “absolute asset value” because these results are based on averages. In the reality, our infrastructure could lead to much worse results or, in a “best-case scenario” case scenario” provided that our organization has unlimited resources or all of our infrastructure is “perfect,” theoretically we can get better results.

  Calculating the time for a mail migration

Scenario 1: Hybrid migration | Single Mailbox Move| Data transfer rate: GB per Hour

The information in the “Performance for migration methods” table says that when using the option of Hybrid configuration for a single mailbox migration, the throughput could be a value starting from 0.3GB per hour and ending with 1GB per hour.

It is quite easy to understand that the estimated throughput of a single mailbox migration is “Inferior” verses the throughput that we achieve when implementing multiple mailbox moves.

If we use a graph for drawing the range, we will get the following graph:

In case that we want to provide an estimate for the time that we need for migration a single 10GB mailbox, the worst-case scenario transfer rate (Low range) is: 32.7~ hours and in the best-case scenario (High range), that time which is required to migrate the mailbox to Exchange Online is:10~ hours.

Hybrid migration | Single Mailbox Move | time range calculation

To make the information more “real”, I have created the following table that includes three optional scenarios for a single mailbox move:

Mailbox Size: 2GB Low Range Mid-Range High Range 6.5~ hours 3.3~ hours 2~ hours Mailbox Size: 5GB Low Range Mid-Range High Range 16.3~ hours 8.2~ hours 5~ hours Mailbox Size: 10GB Low Range Mid-Range High Range 32.7~ hours 16.3~ hours 10~ hours

  What is the calculation formal that we use?

If you are interested in the way that I have to use to calculate these values, the formula is very simple.

For example: the article says, “Observed single mailbox move throughput is in the 0.3–1.0 GB/hour range”.  To be able to calculate the time that will take to migrate 2GB Mailbox using the “Low range” option, we use the following formula.

1GB = 1,024 Megabytes 2GB = 2,048 Megabytes

The “Low range” transfer rate is 300 Megabytes per hour (or if we want to be more precise the number is 306 Megabytes because that article present data that is measured in Gigabytes)

So the formula will be: 2,048/306 = 6.69 is simple words, in case that we want to relate to the Low-range transfer rate in will take six and a half hours to move a Mailbox size of 2GB to the cloud.

Scenario 2: Hybrid migration | Multiple Mailbox Move| Data transfer rate: GB per Hour

In the following chart, we can see the presentation of the information about the transfer rate. The Low-range value is: 10GB, meaning that when we migrate multiple mailboxes to the cloud the worst-case scenario is based on the transfer rate of 10 Gigabytes per hour, the “Mid-range” (the article doesn’t provide a mid-range. I have used the value by calculating the “half-way between the way between the low end value and the High end value) enable us to move of 10 Gigabytes per hour to the cloud in the best-case scenario; we can expect to move of 15 Gigabytes per hour of mailbox data to the cloud.

Example table: hybrid migration Multiple mailbox move (20 mailboxes).

In this example, the calculation is based on the assumption that we use Hybrid migration for migrating 20 mailboxes at the same time.

In this scenario, we add additional variables to the calculation formula because when we deal with a “group of mailboxes” first, we need to define some estimation of the average size of each of the mailboxes and second, we need to calculate the total amount of the data that will be “transferred.” For example, if we want to migrate 20 mailboxes when the average size of each of the mailboxes is 2GB, the total amount of data that will be transferred is: 40 Gigabytes.

For example: in a scenario in which we use Hybrid migration and implement multiple mailbox migrations for mailbox with an average size of 5GB, the time estimation for the compilation of the migration could be as follows:

Low range scenario: 10.2~ hours

Mid-range scenario: 8.2~ hours

High range scenario: 6.8~ hours

Hybrid migration | Multiple Mailbox Move | 20 concurrency

The following table includes estimation of the average time that will take to migrate Exchange on-Premises server mailbox to Exchange Online when we use a migration batch that migrate 20 mailboxes (20 concurrency).

Average Mailbox Size: 2GB (Total size of 40 GB) Low Range Mid-Range High Range 4~ hours 3.3~ hours 2.7~ hours Mailbox Size: 5GB (Total size of 100 GB) Low Range Mid-Range High Range 10.2~ hours 8.2~ hours 6.8~ hours Mailbox Size: 10GB (Total size of 200 GB) Low Range Mid-Range High Range 20.5~ hours 16.4~ hours 13.7~ hours

  Mailbox migration to Office 365 calculator

Based on the data in the “Performance for migration methods” table (Exchange Online Migration Performance and Best Practices  )

I have created an Excel based calculator who will help you to get a general estimation about the time that it will take to migrate existing mail infrastructure to Exchange Online.

The “results” are different when using single mailbox migration versus multiple mailbox migrations, when using different methods of mail migration and when we use a different number of the concurrent mailbox moves.

How to use the Office 365 mail migration calculator

In the following screenshot, we can see an example to the Office 365 mail migration calculator when we use a scenario of Hybrid migration of multiple mailboxes (20 concurrent moves).

We will need to provide the average size of the mailbox (Number 1) and the number of the mailboxes that will be migrated (Number 2). After we enter the required information, we can see the total amount of data that we are going to migrate. In our example, 20 mailboxes with average size of 10,000 MB will create a total of 195 GB (Number 3).

The result defines a time range of passable results (Number 4). For example, in case that our infrastructure is not optimized or overloaded, the time that it will take to migrate 20 mailboxes (each mailbox size is 10GB) is: 19.5 hours. In case that we have “best-case scenario,” case scenario”, the time that it will take to migrate 20 mailboxes is: 13 hours.

  Single Mailbox Migration throughput calculator

Instructions: Enter the value of the mailbox size in MB in the green field and, the result would then be provided a light blue field. This may take a few seconds to load. You may also want to view the full-size “Multiple and Single Mailbox Migration throughput calculator” workbook.

Multiple Mailbox Migration throughput calculator

Instructions: Enter the value of the average mailbox size in MB in the green filed + the number of mailbox that will be migrated and, the result would then be provided a light blue field. This may take a few seconds to load. You may also want to view the full-size “Multiple and Single Mailbox Migration throughput calculator” workbook.

A quick reference for the article series

Office 365 Mail migration | Migration methods | Part 1/4 The first article on this series includes a description of the different mail migration options that are available for us for migrating existing mail infrastructure to Office 365 (Exchange Online) and focus on the features and the characters of the different mail migration methods that relate to the mail migration. Mail migration to Office 365| Factors that impact mail Migration throughput | Part 2/4 The second article includes a review the different factors that impact the performance of the mail migration throughput. Mail migration to Office 365| Optimizing the Mail Migration throughput | Part 3/4 The third article recommendation and best practices for improving and optimize the performance of the mail migration throughput.

Additional reading

How to migrate mailbox data by using the Exchange Admin Center in Office 365

Plan an Exchange Online hybrid deployment in Office 365

Hybrid Deployment Prerequisites

Exchange Online Migration Guided Walk Through (GWT) released

Hybrid Migration Troubleshooter

Microsoft Exchange Online (Office 365) Migration Performance Guide

AnalyzeMoveRequestStats.ps1 script

Office 365 Troubleshooting common migration Issues

Office 365 Mailbox Migration Velocity Best Practices

Now it’s Your Turn! We really want to know what you think about the article