Compose Notes on MongoDB and etcd

Compose Notes are designed to keep you up-to-date on the smaller changes occurring within Compose. In this edition, the deprecation of the slow queries feature and how to use etcd with the new self-signed certificate support.

MongoDB Classic Slow queries now deprecated

The Slow Query service on MongoDB Classic has been deprecated and will no longer be supported. It will be removed from the system on July 16th 2016, exactly three years after its introduction. Slow Query analysis was a promising way of identifying performance issues, but in practice it never performed anywhere near as well as we had hoped and it has become more inaccurate over time. With that in mind, we decided to end its presence in the MongoDB Classic console. Users who wish to monitor their MongoDB Classic query performance should inspect the Live logs.

Going forward with our current production databases, we have made our handling of logs generic so that all databases can have the same options. This means downloadable logs and the ability to ship log data to an analysis server.

For a MongoDB user, that means if you want an environment where you can download your logs, switch your deployment to the current MongoDB. It, like our other databases, offers downloadable log files every 24 hours. If you want to analyse them in closer to real time, with the current MongoDB you can export them using the Syslog-NG add-on to a log analysis service. Services like Loggly and Papertrail provide precision tools for breaking down logs and identifying far more than just simple slow queries.

etcd self-signed Certificates

We've recently made some changes to how we handle SSL with the Compose etcd beta. We have switched to using self-signed certificates to provide more tightly controlled certificate pinning for connecting clients. This does mean there's some changes our customers have to make to their client connections.

Before making those changes, users will need to get their SSL Public key for the server. That can be found on the Compose console for etcd deployments, in the Overview view and below the Connection Strings panel. It's not shown by default, you have to click on the Show SSL Public key and usually enter your password for your Compose account to authorise yourself. Then you will be shown a text block which contains the certificate. You'll have to copy and paste the contents of that text block to a local file to create your certificate. We'll assume you've called that file servercert.crt in the following examples.

To use command-line utilities you'll need to pass the path and filename of that certificate to the utility. Let's start with curl, the rawest way to talk to etcd. Just add the option and parameter -cacert certificate-filename to your command line to get the certificate used:

curl -L https://user:pass@hostname:port/v2/keys/ --cacert ./servercert.crt  

The etcdctl command, which provides a more etcd-centric way to control the system has a similar, but different, option and parameter in --ca-file certificate-filename which would give a command like like:

etcdctl --ca-file servercert.crt --no-sync --peers https://host1:port1,https://host2:post2 -u user:pass ls /  

That parameter can also be set with the value the environment variable ETCDCTL_CA_FILE. Just remember to use an absolute path and filename to point to the certificate when setting the variable.

If you are writing code then how you pass the certificate information will depend on your language and driver. Here's an extract of code for Go using the etcd Go driver. First the imports you'll need:

import (  
    "crypto/tls"
    "crypto/x509"
    "io/ioutil"
    "net/http"
...
    "github.com/coreos/etcd/client"
    ...
)

Now the actual code. Note that peerlist and cafile are strings being passed in from the command-line.

     peers := strings.Split(*peerlist, ",")

    // Read the certificate into a file
    caCert, err := ioutil.ReadFile(*cafile)
    if err != nil {
        log.Fatal(err)
    }

    // Create a certificate pool
    caCertPool := x509.NewCertPool()
    // and add the freshly read certificate to the new pool
    caCertPool.AppendCertsFromPEM(caCert)

    // Create a TLS configuration structure
    // with the certificate pool as it's list of certificate authorities
    tlsConfig := &tls.Config{
        RootCAs: caCertPool,
    }

    // Then create a HTTP transport with that configuration
    transport := &http.Transport{TLSClientConfig: tlsConfig}

    // When we create the etcd client configuration, use that transport
    cfg := client.Config{
        Endpoints:               peers,
        Transport:               transport,
        HeaderTimeoutPerRequest: time.Minute,
        Username:                *username,
        Password:                *password,
    }

    // And create your client as normal. 
    etcdclient, err := client.New(cfg)

You can find this code in the examplco3 repository - that's the final example from our Essentially Etcd series (part 1, part 2 and part 3 where we explore, with code, working with etcd and Go).