As more and more companies use multiple cloud services, large-scale data migrations between clouds have become commonplace. Say Company A, a customer of Cloud X, acquires Company B, a customer of Cloud Y. As the companies merge into one, application data from one cloud provider may need to be migrated to the other to consolidate cloud usage and better manage long-term IT costs.
But moving terabytes or even petabytes of data in and out of different cloud services can cost millions and take many months, sometimes the better part of a year. According to a study led by Experian, only 46% of data migration projects are delivered on time and only 36% stay within budget.
So, what are some key decisions network teams can make to keep massive data migration projects on-time and on-budget?
1. Choose private connectivity over the public internet
Many organizations choose the public internet for data migration, especially for small- to medium-sized transfers. To secure the traffic, network teams typically use VPN tunnels that surround and encrypt the data packets being migrated.
But this level of encryption doesn’t stop governments, Internet Service Providers, or cybercriminals from being able to intercept the data. VPN tunnels are vulnerable to BGP hijacking, which is often used to amplify DDoS attacks.
In addition to the security concerns, throughput is limited on the internet. We’ve all experienced VPN tunnels breaking while we’re working remotely because network traffic is congested. Imagine that happening while moving petabytes of data and you can understand why a large-scale migration could take years to complete.
One of the best alternatives to public internet connectivity is to choose private (or direct) connectivity. This is a dedicated line, with dedicated throughput for your migration. Because the line is private, it’s more secure. Because the throughput isn’t reliant on the whims of public internet traffic routing, you can better predict how long your migration will take.
2. Choose the right throughput
Large-scale data migrations often come down to simple math. If you want to move petabytes of data from one cloud to another as quickly as possible, you’re going to need as much throughput as you can afford.
If you choose to push data through a 10G connection, it’s going to take you ten times longer than it would take if you used a 100G connection. The larger bandwidth connection will cost you more, but it will also significantly cut down the time of your migration, which includes all the additional person-hours spent managing the project, not to mention the services you have to keep running on multiple clouds while the migration is taking place.
Do the math. If the scale of the data migration is big enough, you’ll find that the larger bandwidth connection will pay for itself and then some.
3. Automate as much as possible
APIs can save you plenty of time and money too. With Infrastructure-as-Code (IaC) tools like Terraform and others, network teams don’t even have to log on to the portals of the major cloud providers anymore to light up infrastructure or provision network services. They can write the code and automate the installation and deinstallation of the connectivity they need, when they need it.
Beware of the High Costs of Data Transfer
Egress fees are charged by cloud service providers (CSPs) when data is transferred out of their cloud to another location, like a data center or a competing service provider. These fees are metered, typically in the $0.08-$0.12 per GB range if you use public internet connectivity. Egress fees are typically based on the volume of data being transferred out of the CSP’s network, but they can fluctuate greatly based on the destination of the data, the individual CSP, and several other variables. For example, fees for data transfer between regions (say, Eastern US to Western US) are much higher than if data is transferred within a cloud region. Because of all the potential variables, egress fees can quickly spiral out of control. For some of our customers, egress fees alone can run as high as half a million dollars.
One easy way to lower these fees is to avoid using public internet connectivity to pass traffic, because the clouds charge the highest egress fees for that connectivity method. Use direct connectivity through an API-integrated partner of the cloud provider and benefit from the lower rates clouds charge for transferring data via their hosted direct connectivity services.
How One Customer Saved $826,000 and Cut Migration Time By Five Months
Our customer, a major analytics company, needed to migrate 30 petabytes of data from their legacy cloud provider’s storage to Google Cloud.
They quickly realized that the public internet would not be the best connectivity method for their migration. Migrating that much data via the legacy cloud provider’s internet backbone would have cost our customer more than $1.4 million. The migration would have taken up to a year to complete.
The company chose our cloud router, which supports multi-100G connectivity, to connect the two clouds. The cloud router allows customers to provision multi-cloud connectivity between CSPs within minutes—without requiring any physical connectivity or infrastructure. Backed by the carrier-class Service Level Agreements of our 77+ TB network backbone, the cloud router provided our customer predictable, high throughput for the migration.
The customer used our portal, which provides API-automated direct connections to their CSPs, lowering their egress fees and adding an extra level of security because of our private layer-2 backbone. Our customer was able to complete the migration in five months instead of twelve. We also cut the project cost from $1.4M to roughly $600,000.
Results:
- Cut migration costs by 53%, saving $826,000
- Decreased migration time from one year to five months
- Deployed a fast, flexible, and easy-to-use solution engineered by industry experts
- Leveraged a point-to-point cloud routing solution to instantly create a secure, high-throughput connection
Planning is Key
Large-scale data migration can be a resource-intensive undertaking. It’s vital to make the right connectivity choices at the outset to keep your migration on time and on budget.
Ask yourself these questions before you begin:
- How is your organization currently handling large data transfers between multiple CSPs, between regions of a single CSP, or out of CSPs?
- How much are you projected to spend in cloud egress fees?
- How long do you anticipate your data migration will take?
- Do you have measures in place to ensure data security during your migration?
Need Data Migration Advice?
If your data migration is costing more and taking longer than you expected, chat with our experts about getting your migration back on track.