Working with the New API - Part 1: Go

In this short series of articles we'll be building some utility applications that use the new MongoHQ API, currently in Beta, and guide you through the data you can get out of it. The new API was designed to enable more automation of the deployment process and management of those deployments. Today, we'll build an "Update Reporter" in Go to find out more about the status of your deployed databases.

API Concepts

Although you will likely be familiar with much of the API's concepts – as it maps working directly with MongoHQ's MongoDB database instances to a REST API such as the databases, collections and documents that are part of any MongoDB system – the big addition to the new API is the ability to work with MongoHQ accounts to create, control and delete deployments as needed. This allows the management of many database instances to be automated.

At the highest level, each user has access to a number of accounts and each account has a set of deployments associated with it. A deployment in MongoHQ terms is a set of database instances that go to make up the singular entity that is a database. When you create a MongoDB Elastic Deployment, for example, that creates a deployment with a primary and secondary database instance and a backup database instance, all handled as a single entity and associated with the account's deployments.

The system has been architected to, for example, allow for multiple database deployments in a single deployment, so it may look like there's hoops to jump through which don't achieve much, but trust us, we'll be making use of the API's various enhancement points in the future.

Let's start getting at this information... but before we start anything, we need a Personal Access Token. You get this token by logging into the MongoHQ dashboard, going to My Account and selecting "Manage Personal Access Tokens" on the right hand side of the screen. When a token is generated, you will only see it once, so take note of it at that point. You can create tokens for each of your applications or share one token between applications.

The first application we are going to create is an update report generator. It will find all your databases associated with the deployments under your account and report on what updates they need. We'll be using Go for this example and showing how easy it can be to handle REST and JSON with the language. Lets dive straight into the start of the code, just before where the main() function begins.

var baseurl = "https://beta-api.mongohq.com"  
var token = "<YOUR-PERSONAL-ACCESS-TOKEN-HERE>"  

The baseurl variable points at the URL for the API. Note that this will change in the future when it leaves beta, so make sure you don't hard code it in multiple places in your code. The token variable is for your personal access token. We've put it inline here to keep things simple but ideally you want it stored outside in a secured file or a transient environment variable - make sure it is protected as it gives anyone who possesses it full access your account via the API. (If you think you have leaked a key, go to the Personal Access Tokens page and revoke the leaked key there). Back to the code and the actual start of the program...

func main() {  
    fmt.Println("Update report")

    var accounts []Account

    err := getAndUnmarshal(fmt.Sprintf("%s/accounts", baseurl), &amp;accounts)
    if err != nil {
panic(err.Error())  
    }

After printing an introduction, we create an accounts array which will be used to hold the array of account information which is returned by the List Accounts end point. Where does the Account type come from? We define that ourselves earlier in the code based on the key/value pairs that are documented to be returned. Go has a JSON.Unmarshal function which rather usefully will match the incoming key/value pairs in a JSON document with the variable names within a defined type making it a one-stop shop for decanting that data. All you have to do is to create an appropriate struct type for the data:

type Account struct {  
    Id         string
    Name       string
    Slug       string
    Active     bool
    Created_at string
    Owner_id   string
    Owner      string
}

Using a function, getAndUnmarshal to HTTP GET the JSON results and unpack them into our accounts array. It takes as parameters a URI and a typed variable to be filled in with results and returns an error value.

Unmarshal yard

As we've wrapped the API's versioning and authorization in that function we'll look in detail at getAndUnmarshal now – if you are already familiar with Go JSON functionality you may want to skip forward:

func getAndUnmarshal(url string, result interface{}) error {  
    client := &amp;http.Client{}
    req, _ := http.NewRequest("GET", url, nil)

First , the method gets a HTTP client instance and creates a HTTP request to GET the specified URL. To use the MongoHQ API though we need to set some of the request's headers:

    req.Header.Set("Content-Type", "application-json")
    req.Header.Set("Accept-Version", "2014-06")
    req.Header.Set("Authorization", "Bearer "+token)

The first header sets the content type required to JSON. The second one, "Accept-Version" says that we want to use the "2014-06" version of the API for this request. Finally the "Authorization" header is set to "Bearer " with our access token appended. With the headers set, we can now issue the request and read the response.

    res, err := client.Do(req)
    if err != nil {
return err  
    }

    body, err := ioutil.ReadAll(res.Body)
    if err != nil {
return err  
    }

If there was any protocol error, the first err != nil check would bring this call to an end. Once past that point, you are assured of there being some body content to be read so we then use ioutil.ReadAll to read that into a byte array, body. Although we've got this far, it doesn't mean we've issued a good HTTP request. We need to check if we got a 200 (OK) response. If not, we need to see if there's a JSON encoded error message in the body or whether we should throw an error based on the returned status code:

    if res.StatusCode != http.StatusOK {
var errorMessage ErrorMessage  
json.Unmarshal(body, &amp;errorMessage)  
if errorMessage.Error != "" {  
    return errors.New(errorMessage.Error)
}
return errors.New(res.Status)  
    }

We can now use Go's json library to unpack that content into whatever typed variable was handed to us:

    err = json.Unmarshal(body, result)
    return nil
}

If there's an error unmarshalling that'll just be handed back. With this function we can simplify reading data through the MongoHQ API.

Back to the API

Now we've covered how we query, let's look at what data we're getting from the API. Our first data retrieved was the array of Accounts that the user was allowed to access. Let's start by stepping through those accounts

    for _, account := range accounts {
fmt.Printf("Account name:%s id:%s slug:%s\n\n", account.Name, account.Id, account.Slug)  

First, we'll print out some identifying information for the account. The important part for us, at least for further access, is getting the account slug. This is a string that represents the account and is used as a parameter for many of the API calls. An account can have a number of deployments associated with it so we'll ask the API for that list. We can do that by using the Get all deployments endpoint which is constructed of the baseURL followed by "/accounts/" followed by the account slug and then "/deployments".

        var deployments []Deployment

err := getAndUnmarshal(fmt.Sprintf("%s/accounts/%s/deployments", baseurl, account.Slug), &amp;deployments)  
if err != nil {  
    panic(err.Error())
}

If we do a GET on this URI we get an array of deployment documents and again, we need to define a struct to receive the document data:

type Deployment struct {  
    Id                       string
    Name                     string
    Provider                 string
    Region                   string
    Type                     string
    Plan                     string
    Current_primary          string
    Members                  []string
    Ignored_members          []string
    Allow_multiple_databases bool
    Status                   string
    Databases                []Database
}

Again, lots of information about the deployment, including which region it runs in, what MongoHQ plan its under, what the current primary database is and what the status is. There's also an array of databases that are used in this deployment. Again we leverage the JSON unmarshal in Go and create a Database type for that information to be stored in.

type Database struct {  
    Id               string
    Name             string
    Deprovision_date string
    Status           string
    Deployment_id    string
    Plan             string
}

Extracting Upgrade Information

We now have enough information to start querying each of those databases for their version and update status. This is provided by another endpoint to Get a deployment's database's version infomation. We can step through the deployment's databases and get their version details. This is another nest of structures with the type, version, messages and more upgrade information.

type Version struct {  
    Type                     string
    Version                  string
    Messages                 []string
    Eligible_upgrade_version UpgradeVersion
    Upgrade_path             []UpgradeVersion
}

type UpgradeVersion struct {  
    Upgrade_type string
    Version      string
    Messages     []string
}

But once the structures are defined, its simply a matter of stepping through each deployment...

        for _, deployment := range deployments {
    fmt.Printf("Deployment name/id: %s/%s\n", deployment.Name, deployment.Id)

and then stepping through each of its databases:

            for _, database := range deployment.Databases {
        fmt.Printf("Database name/id: %s/%s is %s\n", database.Name, database.Id, database.Status)

Getting the version information:

                var version Version

        err := getAndUnmarshal(fmt.Sprintf("%s/deployments/%s/%s/version", baseurl, account.Slug, database.Deployment_id), &amp;version)

        if err != nil {
            panic(err.Error())
        }

and printing the results - in this case the database type and version and any messages about available upgrades. Note we could have iterated through the UpgradeVersion data here too for a more complete and detailed report but for now, we just print the associated messages:

                fmt.Printf("The instance is %s version %s\n", version.Type, version.Version)

        for _, msg := range version.Messages {
            fmt.Println(msg)
        }
        fmt.Println()
    }
}
    }
}

And that's pretty much it. If we compile this code (you'll find a full copy here on GitHub and run it you'll get a report on your databases and what you may, or may not, want to upgrade. It'll look a bit like this:

Update report  
Account name:exampleu id:8dba1d4b0ada462e993488e4 slug:exampleu

Deployment name/id: minty/127f95263cc61ac622f7868  
Database name/id: wiktory/8992ecaea59d18f8d0ce0fb8 is running  
The instance is MongoDB version 2.4.9  
This deployment is currently 1 minor versions behind the latest production release of MongoDB.  To upgrade to the latest MongoDB, please clone your database to a staging database to test any upgrades before upgrading a production database.  This will prevent any unexpected breaking changes.

Deployment name/id: /957f40d34c246ff8f07d7262  
Database name/id: exemplum/f9be5e9246fb8a50561d68a9 is running  
The instance is MongoDB version 2.6.0  
This deployment is currently 3 patch versions behind the latest production release of MongoDB.  An upgrade to the latest MongoDB should be non-breaking.  

When you've finished updating your databases you'll get a report which reads more like:

Update report  
Account name:exampleu id:8dba1d4b0ada462e993488e4 slug:exampleu

Deployment name/id: minty/127f95263cc61ac622f7868  
Database name/id: wiktory/8992ecaea59d18f8d0ce0fb8 is running  
The instance is MongoDB version 2.6.3  
This deployment is currently on the latest production release of MongoDB.

Deployment name/id: /957f40d34c246ff8f07d7262  
Database name/id: exemplum/f9be5e9246fb8a50561d68a9 is running  
The instance is MongoDB version 2.6.3  
This deployment is currently on the latest production release of MongoDB.  

Wrapping up

In this first part, we've shown how to navigate through the MongoHQ API, from user to account to deployment to database. There are, of course, many other endpoints in the MongoHQ API which allow you to create databases, rename deployments, view logs or manage backups. In the next part, we'll put a ops navigator together that shows statistical data obtained via the MongoHQ API.

If you aren't a MongoHQ user but want to check out the API, why not sign up for an account and start your explorations today.