Come to Compose: Import Your Data From Anywhere

Update Sep 2016 This article covers importing with Classic MongoDB on Compose. The current MongoDB import process is covered in a newer article, New Import for Compose MongoDB.

Along with the arrival of Rapid Restore, we also took the opportunity to replace the "Clone" option on Compose with a new "Import" feature which could bring data into a Compose MongoDB instance from an internal or external database. We've replaced the old cloning process, copying the entire database - indexes and all, as it's easier and more predictable to carry out the equivalent operation by simply restoring a backup of your current database into a new MongoDB deployment.

In the past, we've been asked for an easy and integrated way to let customers bring in the data they already have on an external MongoDB database into Compose without any intermediate steps. We also anticipated that some users may want to build "clean" databases for analytics and don't want a clone, just a copy of the data. With that in mind, we created the "Import Database" function to replace cloning.

What can import do

"Import Database" is designed to transport the data within a database, rather than the entire content of the database, such as the user/role/password data and indexes information, as the old clone system or a backup/restore would do. The Import process gives a clean, reindexed, fresh database instance ready for reprovisioning to users and applications, for testing new applications or for isolated analysis. There are two sources data can come from and a route for each to travel: for existing databases on Compose there's an import from Compose and for external databases there's an import from remote.

Importing from Compose

Lets start with the simplest import, importing from one Compose database to another. In this example we've got a database called demonstratum in our exemplum deployment and we want to make a copy of all the data into a new database. First, we'll create a new deployment, importexample and create a database called destdb.

import1

Once that's created we click on Admin and select "Import a database" from the right-hand sidebar.

import2

That brings us to the Import Database form...

import3

At the top is a switch for selecting Compose or Remote imports. We want Compose for this operation. Below that is a drop down menu of our Deployments. We wanted data from the exemplum deployment, so we pick that here – well, we don't have to, it's actually the first MongoDB deployment in my configuration so it's already selected. Next, we need to enter the database name we want to import – demonstratum.

Before clicking that Import Database button, pay heed to the warning! Before Import starts importing data, it will first empty the destination database, removing all collections and indexes. We don't have any here because it's a new deployment and database, but be sure you don't try and import into a production database. Ok, now we can click Import Database:

import4

A green flash will appear and disappear saying the import has started and a yellow banner on the "Collections" view will also note that. Reloading this view will give details of the progress and the collections that are already copied or being copied.

import5

When the import is done, revisiting the Collections view will show a green banner saying the copy is complete.

import6

If we look in the "Users" view we can confirm there were no users imported and we can now use our newly imported data.

Importing from Remote

If we set the "Import Database" selector to remote, we get to see the remote import form:

import7

For this to work, we need a MongoDB URI for your database. That means it has to be accessible from the internet and, for your own security, requires usernames and passwords to access it. The URI will require a username and password for your database. So if your getme database is visible on port 10240 on mongodb.example.com and you have an import user with a pasword xyzzy then your MongoDB URI would be

mongodb://import:xyzzy@mongodb.example.com:10240/getme  

Formulate your own MongoDB URI for your database and enter it on this form. Clicking Import Database will start the process and it'll proceed as the Compose import did. Pretty soon, depending on network conditions and your database performance, your data will be on the Compose platform, ready to experience auto-scaling, no-stop backups, the Transporter and more.

Other options

This isn't the only way to get data into your Compose MongoDB database. There's tools like our open source Transporter, our Seed tool (built using Transporter technology) – both of which we'll be writing about soon – and of course the option to write your own import code. "Import Database" is, though, built in, easy to use and at your fingertips.