A How-To for Migrating from MongoDB Classic to Compose for MongoDB

Published

We mentioned back in July that we're on the path to retiring MongoDB Classic on Compose. Since that article came out, we've had a lot of customers ask about how to move their data to Compose for MongoDB from MongoDB Classic. In this article, we show you how to import your data with a click of a button.

There are plenty of reasons why you should move your data from our older deployments to Compose for MongoDB. We outlined most of these in our article on what you need to know about Compose for MongoDB. Essentially, it comes down to Compose for MongoDB having a better architecture, SSL out of the box, oplog access, and a simpler way to import data.

But importing data wasn't simple enough.

So, we developed a tool within the Compose UI to help you import your existing MongoDB Classic data over to Compose for MongoDB. It's designed so that you can easily import your data over to Compose for MongoDB by either importing it all at once or by tailing your MongoDB Classic databases. If you are anticipating a lot of writes to your databases, tailing will make sure that new data that appears in your classic deployment will also appear in Compose for MongoDB. Therefore, when you're ready to switch connection strings to your new deployment, all your databases will be current and ready to go.

In this article, we'll walk you through the import process by migrating a database from MongoDB Classic over to Compose for MongoDB. Let's get started ...

Creating the Deployment and Database

When selecting a Compose for MongoDB deployment to migrate your data to, you have a number of versions that are available to you that are listed in our databases versions documentation. The preferred version on Compose at the time of writing is 3.6.8.

Once you've decided on a version, you'll need to create a new deployment by selecting the Create Deployment button at the top of your Compose console. Once you've selected that, it will take you to the New Deployment panel where you'll select MongoDB. This will open a panel where you can rename the database, select the database region and cloud service provider, select SSL access and the WiredTiger Storage Engine, and choose your database version. SSL access is enabled by default, and you can read about whether the WiredTiger Storage Engine is best for your use case in our documentation or more in-depth from the MongoDB documentation.

Once you're done with the configuration, select Create Deployment at the bottom of the panel. This will kick off the process to create your Compose for MongoDB database.

Creating a database and users

When your deployment has finished provisioning, you will have a new Compose for MongoDB deployment that you can start migrating your data to.

From this new deployment, you'll need to create a database or multiple databases depending on how many databases you have in your MongoDB Classic deployment. Importing a database from MongoDB Classic to Compose for MongoDB is a database-by-database process, not deployment to deployment. Therefore, each database that you want to migrate from MongoDB Classic needs to have its own database set up in your new Compose for MongoDB deployment.

You will also need to recreate users in your new deployment for each of those that you have set up in MongoDB Classic. While all the data from the classic database will be imported to the new database, users will not be imported. So make sure to recreate any users you had in your MongoDB Classic deployment.

To create a database in the Compose console, select Browser from the left-hand menu which will take you to the Database view then add a database name. We're calling our database "example", which is the same name as the database in our MongoDB Classic deployment.

After the database is created, it will take you to the database's administration view. From here, let's show you how to start importing your data from MongoDB Classic.

Importing the Data

Now, select Imports from the left-hand menu. This will take you to the Imports view. Click on the link Start importing data.

This link will take you to an import configuration view where you have a few options to select from.

From this view, you can import from a Compose managed database or a remote database. Since you're importing a MongoDB Classic database, select "Compose" - it's selected by default. From here you have two options: Enable Tailing for deployments where you want to sync writes from a MongoDB Classic database to Compose for MongoDB database over a period of time. Or, if you don't select enable tailing, you can import your data once, which will bulk read and write your data a single time to a Compose for MongoDB database.

We'll show you both options.

Syncing your Database Over Time

There's a checkbox in the Imports view called "Enable Tailing". Tailing is particularly useful if you are anticipating a lot of writes to your MongoDB Classic database while you're migrating. What this does is after the initial bulk import of your MongoDB Classic data to your Compose for MongoDB deployment, the new database will continue to read the oplog of your MongoDB Classic database for up to three days, or until you cancel the process. After that process has ended, your new database will have the same data as your MongoDB Classic database.

With Enable Tailing selected, you'll need to select your MongoDB Classic deployment and specify the database in the classic deployment that you want your new Compose for MongoDB database to tail.

After that, click on Import Database at the bottom of the Imports view. This will take you to another Imports view where you'll see something like the following:

Under "status" you'll see "Starting" initially. This means that the import process is starting and during that time your Compose for MongoDB database is reading and writing the data from your MongoDB Classic database. This includes all of the data and the collections within the specified database.

After that's completed, you will see an update to the status "tailing".

Now, if any writes are made to your MongoDB Classic database, they will be carried over into your new Compose for MongoDB deployment.

There is a caveat. Only new data will be read and written from the MongoDB Classic oplog. If a collection is created with new data, that collection will not be added to the new Compose for MongoDB database during the tailing process. If you want to include a new collection, you'll have to either wait and add it to your new Compose for MongoDB database, or you'll have to cancel the tailing operation and start again.

As mentioned previously, tailing occurs for only three days, or until you decide to cancel the tailing operation. Once three days have passed, we will cancel the tailing process and you will see "Cancelled" as the import status.

Importing Data One Time

If, for any reason, you just want to import the data from the MongoDB Classic database to Compose for MongoDB without tailing, you can do that, too.

From the Compose for MongoDB's Imports view, select the MongoDB Classic deployment from the dropdown menu under "Deployment".

Then add the database name of the MongoDB Classic deployment you want to migrate your data from into your new "example" database.

After that, you can click the Import Database button to import and you'll see the same status as "Starting". Then when the migration is complete you will see "Complete" as the status.

Porting your Application

Whether you choose either option, you should run a few checks to make sure that your data has been successfully migrated. These include data validation and structural compares. At the same time, since you will be using a much newer version of MongoDB, you'll need to make any necessary changes to your application that supports that database version. Consult the MongoDB release notes for database changes by version.

Once you've checked, and double checked that your data is successfully stored in your new Compose for MongoDB database and that your application supports the much newer database version, you can swap out your MongoDB Classic connection string with your new Compose for MongoDB connection string. At that point, all your new reads and writes will take place in your new deployment, and you can deprovision MongoDB Classic.

Summary

Importing your data from MongoDB Classic to Compose for MongoDB is only a start. With lots of enhancements and added security and database features, you'll thoroughly love working with Compose for MongoDB. Compose makes it easy for you to choose which import option is best for you without disturbing your production databases while benefiting from all the improvements that MongoDB has to offer.

If you have any questions about migrating your data and deprovisioning your MongoDB Classic database, feel free to contact us at support@compose.com.

attribution Barth Bailey

Abdullah Alger
Abdullah Alger is a former University lecturer who likes to dig into code, show people how to use and abuse technology, talk about GIS, and fish when the conditions are right. Coffee is in his DNA. Love this article? Head over to Abdullah Alger’s author page to keep reading.

Conquer the Data Layer

Spend your time developing apps, not managing databases.