When importing or restoring your database into MongoDB, make sure that you don't use the admin database. Why? We'll show you.
At Compose, our support team helps customers with a variety of issues related to their deployments. One issue that comes up frequently is related to errors that developers receive when importing or restoring documents inside a new MongoDB deployment and forgetting not to use the admin database.
Whether you're familiar or not with the pitfalls of putting your data into admin, this article will show you some of the errors that you could run into, and show you the easiest way to solve this issue (hint: just don't use the admin database for your data).
The admin database
When creating a Compose MongoDB deployment, the database that is initially created is admin, which lives on different servers where your other databases will live. Although admin is shown as the initial database in the connection strings, it's important to realize that this is the database for administration. If you use the admin database directly you'll run into problems.
The main purpose of this database is to store system collections and user authentication and authorization data, which includes the administrator and users' usernames, passwords, and roles. Access is limited to only to administrators, who have the ability to create, update, and delete users and assign roles.
One of the major security risks of using the admin database is that all users who have access to it have the ability to create, read, update, and delete system users and databases in your deployment. While this can be limited by creating custom roles, it's far better to eliminate the risk altogether. If you manage administrators and users using the Compose console, you will only be able to create administrators within the admin database, and you can manage individual database owners through each database's administration console.
In addition to the security risks, putting data into admin may lead to unforeseeable consequences that will lead to significant problems when importing or restoring data into MongoDB. Let's take a look at some of these problems and what we can do to avoid them.
mongoimport to import into admin
mongoimport command to insert data into the admin database can create some potentially strange side effects. Take for example the following command to import a CSV file containing a list of 550 banks:
mongoimport --host aws-us-west-2-portal.2.dblayer.com --port 15983 --db admin --ssl --sslAllowInvalidCertificates -u admin_user -p adminpass --type csv --headerline --file bankfile.csv --authenticationDatabase admin
As you can see, we are using the admin database along with the administrator username and password along with other security options and flags for importing CSV files. Once the command runs, we'll get the following output from the terminal:
2017-05-04T06:22:40.656-0700 no collection specified 2017-05-04T06:22:40.656-0700 using filename 'bankfile' as collection 2017-05-04T06:22:41.050-0700 connected to: aws-us-west-2-portal.2.dblayer.com:15983 2017-05-04T06:22:41.349-0700 error inserting documents: Writes to config servers must have batch size of 1, found 550 2017-05-04T06:22:41.349-0700 imported 550 documents
Although we tried to import the documents, we ran across
error inserting documents: Writes to config servers must have batch size of 1, found 550. That's because config servers do not accept bulk writes. Even though it states that it has
imported 550 documents, logging into the deployment will show you that in fact no data has been imported into admin.
The solution to this problem is to swap out
--db admin for a new database
--db banks, which will be created automatically when the command is run. Additionally, since we don't have any users who are authorized to write data to the banks database, we'll have to use the
--authenticationDatabase admin option that will allow us to log into our deployment and import the data using the administrator credentials.
So, for example, using the same credentials and only swapping out the database, we'd get:
mongoimport --host aws-us-west-2-portal.2.dblayer.com --port 15983 --db banks --ssl --sslAllowInvalidCertificates -u admin_user -p adminpass --type csv --headerline --file bankfile.csv --authenticationDatabase admin 2017-05-04T06:18:03.591-0700 no collection specified 2017-05-04T06:18:03.591-0700 using filename 'bankfile' as collection 2017-05-04T06:18:03.980-0700 connected to: aws-us-west-2-portal.2.dblayer.com:15983 2017-05-04T06:18:04.313-0700 imported 550 documents
Now, logging into our deployment, we'll see that the database banks has been successfully created and contains 550 documents.
mongos> use banks mongos> db.banklist.find().count() 550
mongorestore to restore data into admin
If you've used
mongodump to download your database into a dump file, you can restore your data along with users and their credentials into your new deployment. Sometimes, however, you may forget to swap out "admin" when copying and pasting your connection information. If you forget to change the database name, it can cause some unwanted problems.
mongodump to get all of your data as well as a copy of the users of that database, you'll have to use the
-dumpDbUsersAndRoles option, which requires an administrator. Since we are using administrator credentials, we'll also have to use the
--authenticationDatabase flag to log in. We might write the following command to dump our bank data:
mongodump --host aws-us-west-2-portal.2.dblayer.com --port 15983 --ssl --sslAllowInvalidCertificates -u admin_user -p adminpass --db banks --dumpDbUsersAndRoles --authenticationDatabase admin
Now that we have the data in a dump file, let's see what happens when we restore it to the admin database using
mongorestore. Notice that we've used the
--restoreDbUsersAndRoles, which states that we want the users and roles of the banks database to be restored:
mongorestore --ssl --sslAllowInvalidCertificates --host aws-us-west-2-portal.2.dblayer.com --port 15984 -u admin_user -p adminpass --restoreDbUsersAndRoles --db admin dump/banks
The problem we'll encounter when running the restore command is the following:
2017-05-04T06:52:24.589-0700 Failed: cannot use --restoreDbUsersAndRoles with the admin database
In this case, MongoDB has stopped any potential problem and just tells us that with the
restoreDbUsersAndRoles flag, we are not able to add new users or data to admin. However, let's see what happens with we remove
2017-05-04T06:55:28.728-0700 building a list of collections to restore from dump/banks dir 2017-05-04T06:55:28.864-0700 reading metadata for admin.banklist from dump/banks/banklist.metadata.json 2017-05-04T06:55:28.914-0700 restoring admin.banklist from dump/banks/banklist.bson 2017-05-04T06:55:29.182-0700 error: Writes to config servers must have batch size of 1, found 550 2017-05-04T06:55:29.182-0700 restoring indexes for collection admin.banklist from metadata 2017-05-04T06:55:29.223-0700 finished restoring admin.banklist (550 documents) 2017-05-04T06:55:29.223-0700 restoring users from dump/banks/$admin.system.users.bson 2017-05-04T06:55:29.288-0700 roles file 'dump/banks/$admin.system.roles.bson' is empty; skipping roles restoration 2017-05-04T06:55:29.288-0700 restoring roles from dump/banks/$admin.system.roles.bson 2017-05-04T06:55:29.678-0700 done
As MongoDB tries to restore the data, we run into the same error that we received when we tried to import the data earlier using
error: Writes to config servers must have batch size of 1, found 550. So, again, MongoDB prevents us from bulk inserts, which will not restore any of our data or users into admin.
Don't drop with admin
The real problem comes when attempting to use the
--drop flag when restoring your data using the admin database. In the previous examples, we saw some of the failures that MongoDB shows, but these don't have an effect on our database, except for the error we'll receive that will reject the data from being restored.
According to the MongoDB documentation:
When the restore includes the admin database, mongorestore with
--dropremoves all user credentials and replaces them with the users defined in the dump file.
Therefore, what will happen is that all of our administrator credentials will be deleted. At the same time, our deployment will fail and we'll see the errors when we log into our deployment's console on Compose.
--drop flag to the
mongorestore command will look like:
mongorestore --ssl --sslAllowInvalidCertificates --host aws-us-west-2-portal.2.dblayer.com --port 15984 -u admin_user -p adminpass --db admin -drop dump/banks
Now, after running the command, let's try logging into our deployment. If you login, you'll immediately see something like:
MongoDB shell version: 3.2.11 connecting to: aws-us-west-2-portal.2.dblayer.com:15984/admin 2017-05-04T07:10:50.231-0700 W NETWORK [thread1] SSL peer certificate validation failed: unable to get local issuer certificate 2017-05-04T07:10:50.388-0700 E QUERY [thread1] Error: Authentication failed. : DB.prototype._authOrThrow@src/mongo/shell/db.js:1441:20 @(auth):6:1 @(auth):1:2 exception: login failed
Since all the deployment's users have been dropped, our authentication credentials have also been deleted. This is also shown when logging into our Compose deployment at the top of the browser window like the following:
At this point, you'll have to delete your MongoDB deployment from the console and create a new one since you won't be able to access your databases from the browser.
To avoid the problem of dropping your administrators, and to restore users that were stored in your dump file, let's create a new database called newbanks using the
--db flag. We'll also use the admin database for authentication like we did when using
mongoimport earlier. At the same time, since we are now using newbanks as our database, we can use the
restoreDbUsersAndRoles flag, which will import the user credentials that were dumped from the banks database. This will look like:
mongorestore --ssl --sslAllowInvalidCertificates --host aws-us-west-2-portal.2.dblayer.com --port 15983 -u admin_user -p adminpass --drop --authenticationDatabase admin --restoreDbUsersAndRoles --db newbanks dump/banks
As you may have noticed, we are using the
--drop flag here. This will make sure that when creating newbanks we are dropping any collections in the database that might be in the target deployment and restoring all of the data and user credentials as a fresh copy.
Create new databases with specific owners and users
To avoid using the admin database, it's best to create databases that you can connect to which have their own users and roles assigned to them. To do this from the Compose console, click on the Browser link on the Overview view, then click the Add database button on the top right of the Browser view.
After clicking on the button, fill in the field next to
use with the name of a new database. In this instance, we will call it "my_data". Then click run to create the database.
After the database has been created, we can create database users or collections. However, to get the connection strings for the database, we can click the Admin button on the database menu.
This will provide us with the connection strings that we need to connect to our database with any application. As you can see, all of the connection information is the same as on the Overview view, except instead of admin as your database, you now have the connection strings using to your new database my_data.
Adding users to the database is also done by clicking on the Users button on the menu. There we can click the button Add User.
This will prompt us to add a new username and password for the database administrator, or
dbOwner, who has administrative rights to the database. In this example, our user is "superman"
Once the user has been created by clicking Run, you will see that "superman" is now the owner of the new database.
readOnly privileges to users, you can create a user and click the readOnly button under the
db.addUser command. We'll create the user "batman" who will only have
This will give the new user "batman" only read privileges for this database, which is shown next to their name once he has been added to the database.
In this article, we showed you some of the pitfalls of using admin, but we also showed you ways to rectify the most common problems you're likely to encounter. Just remember, when setting up a new Compose MongoDB deployment, be aware that when trying to import or restore data from an existing database, always make sure to not use the admin database. By keeping your deployment's users separate from the core data, you'll save yourself from some serious headaches that can be avoided.
If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at firstname.lastname@example.org. We're happy to hear from you.
attributionAnthony Da Cruz