Bringing SSL and more to Compose MongoDB

TL;DR: There's a new MongoDB deployment option going into beta at Compose and it brings SSL connectivity and a new flexible configuration to the table.

Over the last year, since introducing MongoDB Elastic Deployments, we've been working on the next generation of MongoDB deployments. We'll be calling this MongoDB+ on Compose until it leaves beta when it will become our main MongoDB offering. MongoDB+ has involved some pretty extensive re-architecting of our deployment model but the result is a system that can deliver, one of the most requested features, SSL connectivity to the MongoDB database. This has been made possible by bringing MongoDB to the Compose platform we have been using for other databases, unifying how we deliver databases to you.

To bring effective SSL to the configuration, we've had to change how we configure MongoDB. We previously used un-sharded databases with a primary and secondary data members forming a replica set, all of which faced onto the Internet to enable customer connectivity. This has been a very useful configuration, but it stopped us from adding SSL to the set up. The proxy would not be sufficiently smart to safely terminate the incoming SSL traffic and pass it on to the correct member of the replica set.

Routing and SSL

Our new solution involves using MongoDB's sharding technology, but without requiring that you shard your database, although you will be able to shard and add shards in the future. What this arrangement gets us is the ability to use the mongos router. The router in a sharded system works with config servers to get requests to the appropriate shard in the cluster.

In terms of access, the router becomes a fixed point in the network to connect to, rather than trying to select which database member you want to connect to. This makes querying the database so much simpler as you only need to know the address of the routers to access it rather than the topology of the replica set which, if the setup was reconfigured, would also have to be changed in your applications.

We set up the MongoDB+ configuration with two of these routers. Now, we could configure them with MongoDB's SSL but that would mean that if there was an issue with SSL security we'd have to wait on a MongoDB update to get the problem resolved. Given the preponderance of SSL issues in the last couple of years, we want to be more agile than that.

With that in mind, the Mongos router capsules also play host to haproxy which serves as the endpoint for incoming SSL connections. We have more control over how the haproxy modules are built and configured and we can use them operate black or whitelist IP address filtering and other access controls. We can also turn off the SSL where people don't feel they need it. The transit between the proxy and the mongos router is within the capsule which makes it quick and secure.

MongoDB SSL vs private VLANs

This isn't the only reason why we terminate SSL before we get to the router. In MongoDB's architecture, if a member of the cluster has SSL enabled, then it performs all its networking with SSL enabled including talking to other nodes. This design choice makes sense if you are building a cluster with its members scattered around the cloud, but if the connections between the members are already secured then it just means a lot of redundant encryption taking place.

With the Compose platform, all the network traffic is handled by our Software Defined Networking/Private VLAN technology. That means that all the members of the deployment are configured to live within their own private VLAN distributed over the Compose cloud. This is the same SDN technology we've deployed with Redis, Elasticsearch, PostgreSQL and RethinkDB and talked about before. Because it's already in an private VLAN, MongoDB's member to member SSL is unnecessary, and by handling incoming SSL connections before we reach the router, we ensure we don't have add redundant encryption to satisfy MongoDB's design.

Router to Replica set

The mongos routers talks, with the assistance of three configuration servers, to the replica sets that contain the shards. In our basic configuration though, we only have the one replica set and, therefore, one shard. Our scaling system allows us to raise, and lower, resources allocated to this replica set to keep it elastic. We should mention that our replica sets actually have four members; two for data, one for arbitration and one for backup. The latter backup member doesn't participate in the voting for the replica set, which is why there's a member for arbitration to help break ties between the two data members.

Other configurations could, of course, include multiple shards, but our traditional warning to avoid premature sharding still stands. Sharding your dataset is, practically, a one way trip and without a good understanding and experience of how your proposed keys will behave it will be road with many good intentions paving it. Get as much information as you can during development and early production before even considering embarking on sharding your database. We will, in the future, be offering the ability to add shards to your configuration and shard your dataset.

Additionally...

While we default to two router capsules, we can add more to allow for more incoming connections or redundancy. There's also another way to access your deployment in the form of the SSH access portal. This is an optional capsule that can be used to get an SSH tunnel into the private VLAN. This allows an application full visibility of all the members within the deployment to get oplog access. We'll be looking into this in more detail when we cover oplog access with Compose MongoDB+.

Update-Dec 2015: Since this was written we've come up with better ways to provide oplog access which don't require SSH tunnels and are working to make them generally available. If you have different requirement for SSH tunnel access, please contact support.

With MongoDB+ running on the Compose platform's capsule architecture, the way is open for a whole range of add-ons and extensions. We actually offered a taste of one of those options recently with the launch of the New Relic add-on.

Total Control

In the past, we've had to limit the access of users to the levers of control within MongoDB. This was done so we could ensure that our MongoDB installations were as stable and predictable as possible. But now with our private VLANs and capsule based architecture, we can safely hand over the admin keys to users and allow them to create their own privileged users.

In Beta Now

MongoDB+ on Compose is now in beta and you can get access to it by logging into your Compose account, going to the Deployments page and clicking the Add Deployment button in the top right. We're setting the current pricing for MongoDB+ at $31 for the first gigabyte of storage and $18 for each subsequent gigabyte. If you have any queries about MongoDB+, whether it be usage, features or capabilities, just contact support.