Hot-migrating datacentres – we did it!
Upgrading from versions that are no longer supported, extending or replacing a datacentre, implementing a disaster recovery plan — there are many reasons for moving workloads between different datacentres. And it has never been so quick and easy to switch from the US West Coast to the East Coast or from Amsterdam to Limbourg. In just a couple of clicks, workloads can be sent between different datacentres via secure HCX tunnels.
Here are a few figures to demonstrate the efficiency of this system: it took a customer five weeks to move 300 TB of VMs, including planning, setup, replication and switchover. In one day, this customer increased the volume of data transferred between two datacentres in Germany to 23 TB, or around 1 TB per hour. Another customer moved more than 200 TB of data from their datacentre — spread over 750 VMs — without downtime.
Until a year ago, no-one would have imagined that hot-migrations could be possible. Even the idea of migrating workloads between two datacentres seemed unattainable, so hot-migrations were just a pipe dream.
Workloads are transferred using a VMware technology called HCX, aimed at the Private Cloud platform. On top of ensuring that workloads are migrated securely, this technology provides a seamless transition by establishing a network connection between the source and destination datacentres via a Layer 2 stretched network. Under standard conditions, a virtual machine that is sent “hot” in the Private Cloud does not lose connectivity with the other machines it operates.
HCX uses three appliances. One is the Cloud Gateway (CGW), for managing the transfer of virtual machines from one datacentre to another. The second, the WAN Accelerator, works in conjunction with the CGW. The third serves the stretched network (L2C). These appliances are automatically deployed on the Private Cloud side, and require a fourth appliance on-premises to control the deployment and configuration of these three essential elements.
Note: The CGW also appears in the inventory as a registered host.
In short, at least two tunnels are built between the source datacentre and the destination datacentre, the OVH Private Cloud. One tunnel runs between the CGWs to transfer the VMs. One tunnel goes between the L2Cs to create a stretched network in case a subnetwork needs to be extended. It goes without saying that it is possible to deploy several L2Cs, depending on the number of networks that need extending.
Once the architecture is deployed on-premises, a dashboard summarises the migration possibilities and provides a log.
There are several ways to move virtual machines: a "hot" migration, a so-called "warm" migration and a "cold" migration.
Hot-migration is by far the most impressive. In just a few clicks, a virtual machine appears in the destination datacentre without losing its state, connectivity and context. The process is similar to vMotion, which VMware users already know about. It is called vMotion Migration. The idea is that the VM datastore is sent to the destination datacentre. Once the VM backup is fully synchronised, the memory and CPU are in turn synchronised and the destination datacentre takes over. This method has a limitation: the sequencing of the process can impact workloads that are distributed over several VMs and require low latency between VMs. After migrating a VM, there is a noticeable increase in latency between the datacentres.
We address this issue using "Bulk Migration", which also adds features to help with controlled migration. The aim is to synchronise one or more VMs with the destination datacentre while they are still in the source datacentre, and to maintain this synchronisation over time. The switchover of all VMs takes place at a time chosen by the administrator who initiated the migration, in a timeslot most favourable for the switchover. The switchover simply involves switching off the VM in the source datacentre and booting the VM in the destination datacentre. As well as controlling the switchover period, it is possible to customise the VM (update VMware Tools, upgrade virtual hardware, etc.). Since all VMs move at the same time, there are no issues with latency in the stretched network.
The last migration method is disruptive for production servers, and is therefore more intended for the migration of templates, archives and backups. It is done cold, with the VM off, and simply synchronises and switches automatically when all the data has arrived at the destination datacentre.
We have more than three years’ experience in secure workload migration, initially with vCloud Air datacentres, then with OVH's own datacentres. During this period we have migrated exabytes of data all over the world.
HCX is a tool designed to address several migration-related issues, including the operational maintenance of virtual machines during transfer, the need to switch sets of VMs and the network connection between different datacentres. It requires some work on the architecture. A migration is in fact prepared upstream, with sufficient sizing of the destination datacentre. This can be adapted over time on the OVH Private Cloud. It is also necessary to work on an estimate of the switchover duration and strategy for the different workloads in the source datacentre. As for the rest, it's just a few clicks in HCX.