Editor’s note: This post was originally written for the Blue Medora blog. You can check out the original here.
One-size-fits-all doesn’t work. When it comes to IT infrastructure, it’s one-size-fits-none.
Whether on-premise or in the cloud, data centers are complicated, and no two are alike. As your organization’s needs change, your data center infrastructure size should change with it.
One of those changes that force you to reconsider your infrastructure is when you migrate your database. Don’t expect your infrastructure to successfully support a complete database migration without any changes to its size.
Your infrastructure’s size could increase or decrease, but sizing it correctly is important. And if you don’t adjust your infrastructure to fit your migration needs, you’ll face impending project failure. To help ensure your success, we’ll cover how to appropriately size your infrastructure when migrating your database. And we’ll discuss how to pick what’s right for today, as well as how to plan for future success.
Migrations Are In
With the proliferation of the cloud, many organizations have made the decision to migrate their database from an on-premise solution to a cloud provider. Others have even decided to move from commercial database solutions, like those from Oracle or Microsoft, to open-source databases, such as MySQL and PostgreSQL.
And in the process, those who were running these databases on physical machines are taking the opportunity to move to virtual machines instead.
Are you considering such a migration? If so, here are some things you’ll want to think about.
Database Migration Considerations
When migrating a database, there are a number of things you’ll want to evaluate to ensure your success. You’ll have to think about the people it will involve, the time it will take, and how your database will perform in the new environment. All that is important. But one of the first things to think about is the infrastructure and how you’ll size it to accommodate this migration.
While you may wish it did, your organization’s databases obviously don’t run on air. You need the server, whether physical or virtual, and you need the network connectivity and its associated devices.
When migrating a database, there are three things you should consider that encompass the above and impact infrastructure sizing: migration cost, migration duration, and post-migration cost. Let’s talk about these three considerations in more detail.
1. Database Migration Costs
When migrating a database, the cost is usually top of mind. That’s obvious. Nobody’s actually handing out free databases, even when the price is zero.
But you also need talented professionals who will help complete the migration, and you need the hardware and software components required to run your database. Plus, depending on the size of your network, you may have to buy more bandwidth, at least temporarily. All that costs money and must be accounted for.
2. Migration Duration
Doing a database migration involves making changes to the database and the application that uses it. You may need to refactor your code and make changes to the schema. That can take weeks or months, depending on database size. So you need to account for the time it will take to complete the migration itself.
Are you moving from a database in a physical environment to a virtual one? Or perhaps you’re going from a commercial database to an open-source one. Preparing for that change will take time. If you’re moving from one colocation to another, you’ll need to account for the migration duration—remember, you might not be able to leave your current colocation without extending a contract. More costs!
3. Post-Migration Costs
Naturally, you’ve considered the cost of actually doing the migration itself, as I discussed above. But what happens after the migration is done and your database is running in its new environment? What will that cost to run? This needs to be considered, too.
Moving to the cloud or moving from physical to virtual is supposed to reduce costs. But if you move your large database and are paying per resource consumed, you might be in for a rude awakening with the cost of maintaining your database post-migration.
Sizing Your Infrastructure for Migration
All of the considerations I mentioned above will affect how you size your infrastructure to support your database migration.
Identify Your Needs Today
In order to have a successful migration and properly size your infrastructure, you need to do an analysis of what your needs are. You already have an existing database running on existing infrastructure. You must now identify what you’ll need in the new environment to support migrating your database today.
Here are three ways you can do that:
1. Determine database size. You first need to know how your database behaves in the here and now. What resources does it consume? What’s its size? You’ll want to know since a smaller database size usually makes for a faster migration to your new database. Also, how many database tables does your source database contain? What about the number of schemas? If you’re migrating to a different database, say from Oracle to SQL Server, you’ll need to convert the schemas. The number of schemas will play a part in determining how fast copying your data will take. Or what if a decision’s been made to move to an in-memory database in the target database? The size of your database will impact your ability to do this without making specific changes to your database itself or your database infrastructure post-migration. So knowing answers to these types of questions will help you estimate how long it will take to copy data from the source to your target database.
2. Understand your applications. Your database is serving data to an application that’s located in the same data center (or somewhere nearby). It could even be providing data to many applications. Do you understand the architecture of how these applications communicate with your database? How do these applications process information read from your database? If a monolith application that talks to your database moves to the cloud, it’ll likely get rewritten and broken up into microservices. One of these services could be a local database cache on an application server to reduce database reads and writes across the network. If this local cache changes to a remote cache service when in the cloud, you now have chatty database communications across microservices, leading to increased latency and performance issues. So you’ll need to understand that application to determine whether a remote versus a local cache makes sense after the migration. Knowing this type of information about your applications may lead you to beef up your infrastructure to properly support it all.
3. Know the network. In the age of DevOps, the network seems sometimes to either be an afterthought or a nuisance. But without the network, there’s no communication, even if you’re all virtual. How are you migrating the data stored in the database? Unless you’re doing a “lift and shift”, this data will traverse the network. You’ll need to ensure that the bandwidth capacity in the existing environment can support the transfer without impacting other applications. And what about firewalls? If the database is migrating to the cloud, you may need to open up ports on those. Are your network devices heavily utilized today? Can they handle all the data going through them for the migration? You need to answer these questions because you may need new or upgraded bandwidth, firewalls, and switches to support the migration.
Plan For Tomorrow
Being proactive is something every IT leader wants, but in my experience, it doesn’t often happen. Well, this migration is your opportunity to be proactive.
Now that you’ve looked at what you need today to size your infrastructure when migrating your database, be sure to also look at what you will need on day two after the migration, as well as six months into the future. Day two to me includes the ongoing operational things required to manage your database post-migration. What happens when data starts traversing the new environment and usage goes up? Have you sized your infrastructure to handle this increase? Or will performance suffer?
In order to plan for day two, you need to do some planning to determine database growth. Is the database size trending upwards? When that happens, do you have enough CPU, storage or memory resources to support it?
If you’ve moved to the cloud, you shouldn’t have to worry about whether or not your resources will support it, especially if you’ve done a DBaaS migration. You’ll be able to upgrade fairly easily. After all, the flexibility to scale up as needed is a reason to go to the cloud.
But if you’ve done a “lift and shift” database migration, where you’ve moved your database as-is to cloud infrastructure, scaling up may not be as simple a decision. One reason for that is because, in a DBaaS, you’re paying for largely your data on your service provider’s database. Your costs can start off lower and grow over time.
In a “lift and shift,” you’re moving your own database, with any of its existing components and issues, to your service provider’s VM. Scaling this could prove to be not only more problematic due to any unaddressed issues premigration; it could also be more costly since you’re paying for resources used immediately rather than only growth over time. So you must account for the cost of increasing the resources your database needs, depending on the migration method you use.
The opposite works too. If you’ve crunched the numbers and find that the cost of increasing your infrastructure isn’t manageable, you need to consider making changes to the database during the migrating process. Can you reduce the database size to make it smaller after migration? Does everything need to migrate? This would allow you to reduce your infrastructure size if required.
Monitoring It All
At this point, you might be thinking, “How do I go about answering these questions?” Well, hopefully, you’re monitoring your infrastructure already. You are doing that right?
If you have a bunch of tools for monitoring, consider something that lets you view monitoring data in a single pane of glass.
With that data, you should have information such as how much CPU and memory your database typically uses, as well as how much traffic is going across your network, over what devices, and to which applications.
From this, I recommend creating a list of infrastructure components and the resources you will need to buy to migrate your database. If your current infrastructure won’t be able to support this migration, this exercise will help you identify where you need additional infrastructure.
Once you’ve gone through and appropriately sized your infrastructure for migrating your database today and you’ve planned for the future, after the migration, to minimize the impact on your infrastructure, you can’t forget to monitor in the new environment. This is especially important if you’ve migrated your database to multiple clouds.
If you’ve done an on-premise database migration, you may be able to utilize the same monitoring tools you had pre-migration. But maybe you’ve gone from a physical server infrastructure to a virtual one. Those tools may not be able to do the job. How will you know whether your database is performing the way it was in the old environment without any monitoring? You’d need a monitoring tool that’s built to monitor cloud databases.
Success Comes to Those Who Plan
Migrating your database will be a challenging endeavor. As I’ve highlighted, there are a number of things to consider and questions to answer. You’ll need to ensure that the database utilizes a right-sized infrastructure to support it both during the migration and after.
The best way to do this is to plan and develop a strategy for infrastructure sizing. This doesn’t mean that everything will go smoothly, as I’m sure you know and have experienced with past IT projects. But having a plan should make migrating your database less painful and more likely to succeed.
And isn’t that what we all want?