Bach - The Compose API at your command(line)

Published

Find out how to use the power of the command line to control your Compose database deployments. The Bach tool is an easy way to harness the mighty Compose API.

The Compose API is a powerful way to create new deployments, back them up, redeploy with restores, get credentials and much more, but you may not have the time or inclination to master the invoking of curl commands and the creation of JSON parameters for those commands. Worry not, we don't either and that's why we created Bach.

Bach is a command-line for the Compose API for those of us who live in the shell. It's written in Go and there's a sister project, gocomposeapi which wraps up the Compose API into something more Go idiomatic. Don't worry about that though as there are prebuilt binaries also available for macOS, Linux and Windows in the Bach releases folder.

Download it and copy the binary somewhere you can use it. Then run bach --help...

$ bach --help 
Bach is designed as simple route to accessing the Compose API

Usage:  
  bach [command]

Available Commands:  
  about       About Bach
  account     Show Account Details
  alerts      Show Alerts for deployment
  backups     Commands for backups
  clusters    Show clusters
  create      Create a deployment
  databases   List databases
  datacenters Lists available datacenters
  deployments Show Deployments attached to account
  deprovision Deprovision a deployment
  details     Show details for a deployment
  help        Help about any command
  recipe      Show details of a recipe
  recipes     Show Recipes related to deployment
  scale       Show scale information for a deployment
  user        Show user information
  versions    Show versions for deployment database
  watch       Watch a recipe status

Flags:  
      --caescaped      Display full CAs as escaped strings
      --fullca         Show all of CA Certificates
  -h, --help           help for bach
      --json           Output post-processed JSON results
      --nodecodeca     Do not Decode base64 CA Certificates
      --raw            Output raw JSON responses
      --token string   Your API Token (default "Your API Token")

Use "bach [command] --help" for more information about a command.  

That's quite a few commands available with an almost one-to-one mapping to the Compose API. Each command has its own help too; just run bach command --help. The easiest way to show what Bach can do is to take you through the process of creating a deployment with it.

The COMPOSEAPITOKEN

Before you do anything with Bach, you'll need your Compose API Token. Although there are some API calls that don't need the API token, Bach insists that you set one for all commands. You can generate an API token in the Compose console, through the Account option. We recommend you create one token specifically for Bach and don't use it for multiple different purposes - you can issue as many API tokens as you need and each one currently has full account owner permissions so be careful with them. Once you have one, you can set it in the COMPOSEAPITOKEN environment variable, or set it through the --token flag; we recommend you use the environment variable.

The first two commands we'll look at are databases and datacenters. These are informative commands, one listing all the types of database available to deploy, the other listing the datacenters where those databases can be deployed. On Compose, we're using the term datacenter not to refer to a single building or facility. It refers to a configuration of zones or networks that we've combined to provide a highly available, resilient entity in a particular geographic region within a specific cloud.

$ bach datacenters
         Region: frankfurt-2
       Provider: softlayer
           Slug: softlayer:frankfurt-2

         Region: london-02
       Provider: softlayer
           Slug: softlayer:london-02

         Region: us-east1
       Provider: gce
           Slug: gce:us-east1
...
$ bach databases
bach databases  
           Type: elastic_search
         Status: stable
        Version: 2.4.4 (stable) Preferred

           Type: etcd
         Status: beta
        Version: 2.3.8 (beta) Preferred

           Type: mongodb
         Status: stable
        Version: 3.2.10 (stable)
        Version: 3.2.11 (stable) Preferred
...

If you have a Compose Enterprise cluster, then bach clusters will deliver the same kind of identifying information for them.

Creating a new database

So, if we want to create a MongoDB in London Softlayer, we can take the slug from the datacenters entry softlayer:london-02 and the type from the databases mongodb and know that, unless we override it, we'll get the preferred version 3.2.11.

Time to introduce the create command. The bach create command requires a name for the deployment and a database type but if we do this:

$ bach create exampledb mongodb
2017/05/15 13:57:49 Must supply either a --cluster id or --datacenter slug  

There's two places a deployment can go, either a private Compose Enterprise cluster or a Compose datacenter, so one or the other has to be specified. We already have the slug for the datacenter so we'll add --datacenter softlayer:london-02 to our command line... but before we move on, let us take a moment and get the help for create:

$ bach create --help
Creates a deployment. Requires deployment name and database type.

Usage:  
  bach create [deployment name] [database type] [flags]

Flags:  
      --cluster string      Cluster Id
      --datacenter string   Datacenter region
  -h, --help                help for create
      --ssl                 SSL required (where supported)
      --version string      Database version required
      --wiredtiger          Use WiredTiger storage (MongoDB only)
...

There are two flags here worth mentioning. The --ssl flag is used to enable SSL on the database deployment where it's available. It defaults to false, so specify it to get SSL support. The other flag, --wiredtiger, only works with MongoDB deployments; use it to enable a WiredTiger deployment complete with more memory, storage, and the latest MongoDB storage engine. We'll add all those flags now...

$ bach create exampledb mongodb --datacenter softlayer:london-02 --ssl --wiredtiger
             ID: 591af872cb404d0015001c9b
           Name: exampledb
           Type: mongodb
     Created At: 2017-05-16 13:02:42 +0000 UTC
 Prov Recipe ID: 591af872cb404d0015001c99
 CA Certificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0t... (Use --fullca for certificate)
    Web UI Link: https://app.compose.io/compose-3/deployments/exampledb
         Health: 
            SSH: 
          Admin: 
       SSHAdmin: 
    CLI Connect: [mongo --ssl --sslAllowInvalidCertificates sl-eu-lon-2-portal.3.dblayer.com:17774/admin -u admin -p VUHUEQFAGFQUUJWW]
 Direct Connect: [mongodb://admin:VUHUEQFAGFQUUJWW@sl-eu-lon-2-portal.3.dblayer.com:17774,sl-eu-lon-2-portal.1.dblayer.com:17774/admin?ssl=true]

That's the initial credentials of the database coming back. You get the admin password and connection strings and more. Here's the important bit though - you get a recipe ID too when you start creating a database.

Recipes and watches

Compose runs "recipes" to manage tasks like deploying a database or, well, anything that will be a change to the deployment. In the create example, see the "Prov Recipe ID", that is the recipe that is managing the provisioning of the database. The command bach watch lets you track the progress of a recipe like so:

$ bach watch 591af872cb404d0015001c99                 master
             ID: 591af872cb404d0015001c99
       Template: Recipes::Deployment::Run
         Status: running
  Status Detail: Running post_create_capsule on mongodb400.sl-eu-lon-2-data.0.
     Account ID: 542da1926b9c7d465d00000a
  Deployment ID: 591af872cb404d0015001c9b
           Name: Provision
     Created At: 2017-05-16 13:02:42.153 +0000 UTC
     Updated At: 2017-05-16 13:02:59.338 +0000 UTC
  Child Recipes: 1

             ID: 591af872cb404d0015001c99
       Template: Recipes::Deployment::Run
         Status: running
  Status Detail: Running post_create_capsule on mongodb400.sl-eu-lon-2-data.0.
     Account ID: 542da1926b9c7d465d00000a
...

That command will keep bach polling every 5 seconds, getting the latest status of that recipe until the Status stops being set to "running". If you don't want to poll for updates, bach recipe 591af872cb404d0015001c99 would pull the current recipe status and display it once. The interesting thing with recipes is that for every Compose deployment, we keep a record of every recipe that has ever been run against it. You can see that list if you pass a deployment id to the bach recipes command. We can see the deployment id of our newly created database just after we kicked the process off; it's the ID field and if we run bach recipes 591af872cb404d0015001c9b we'll get the details of the provisioning recipe thats running, or just finished running. But how to find other deployment ids?

Digging into deployments

That's where bach deployments comes in. It lists all the database deployments associated with the account.

$ bach deployments
             ID: 591af872cb404d0015001c9b
           Name: exampledb
           Type: mongodb
...

Where you have a lot of deployments, you can filter by type with --type (or -t) and a database type, or by name with --filter (-f) and a regular expression. Once you have a deployment id, you can get the deployment's details with bach details:

$ bach details 591af872cb404d0015001c9b
             ID: 591af872cb404d0015001c9b
           Name: exampledb
           Type: mongodb
     Created At: 2017-05-16 13:02:42.239 +0000 UTC
 CA Certificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0t... (Use --fullca for certificate)
    Web UI Link: https://app.compose.io/compose-3/deployments/exampledb
         Health: 
            SSH: 
          Admin: 
       SSHAdmin: 
    CLI Connect: [mongo --ssl --sslAllowInvalidCertificates sl-eu-lon-2-portal.3.dblayer.com:17774/admin -u admin -p VUHUEQFAGFQUUJWW]
 Direct Connect: [mongodb://admin:VUHUEQFAGFQUUJWW@sl-eu-lon-2-portal.3.dblayer.com:17774,sl-eu-lon-2-portal.1.dblayer.com:17774/admin?ssl=true]
$

If we added --fullca, that CA certificate would be decoded and displayed in full. That means with two commands you can programmatically get the full credentials for a deployed database.

Backing up, Restoring out

Backups are also accessible. There's a whole set of bach backups subcommands. If we want to set off an on-demand backup we can use bach backup start:

$ bach backups start 591af872cb404d0015001c9b       ⏎ master
             ID: 591b03b4d9406d0010001cdc
       Template: Recipes::Deployment::Run
         Status: running
  Status Detail: Running usage on mongodb1340.sl-eu-lon-2-portal.3.
     Account ID: 542da1926b9c7d465d00000a
  Deployment ID: 591af872cb404d0015001c9b
           Name: Backup
     Created At: 2017-05-16 13:50:44 +0000 UTC
     Updated At: 2017-05-16 13:50:44 +0000 UTC
  Child Recipes: 0
$

It's that easy to start an on-demand backup. And that information returned - that's a recipe so you can use bach recipe or bach watch to track its progress. Then, you can list the backups on that deployment with bach backups list:

$ bach backups list 591af872cb404d0015001c9b
      Backup ID: 591b03b6c381128381000059
  Deployment ID: 591af872cb404d0015001c9b
    Backup Name: exampledb_2017-05-16_13-50-46_utc_on_demand
           Type: on_demand
         Status: complete

This is a new deployment; older deployments will show all the historical backups available. For now, we've got the backup id we are interested in. We can get the download details of that with bach backups get. This is a slightly different command in that it needs the both the deployment id and backup id as parameters.

$ bach backups get 591af872cb404d0015001c9b 591b03b6c381128381000059
      Backup ID: 591b03b6c381128381000059
  Deployment ID: 591af872cb404d0015001c9b
    Backup Name: exampledb_2017-05-16_13-50-46_utc_on_demand
           Type: on_demand
         Status: complete
  Download Link: https://dblayer-backups-mongodb-shard.s3.amazonaws.com/compose-3/591af872cb404d0015001c9b/exampledb_2017-05-16_13-50-46_utc_on_demand.tar.gz...
$ 

That URL can be fed to wget or curl to retrieve the backup if you want a local copy. You can, though, use bach backup restore - that will restore your backup into a new deployment. The command takes the deployment id, the backup id and the name for a new deployment. It'll also need either a --datacenter or --cluster flag in the same way bach create needs those parameters. You can use the --ssl flag on MongoDB deployments to enable SSL on MongoDB deployments too.

$ bach backups restore 591af872cb404d0015001c9b 591b03b6c381128381000059 newexampledb --datacenter softlayer:london-02 --ssl
             ID: 591b348fd1db8f0014000026
           Name: newexampledb
           Type: mongodb
     Created At: 2017-05-16 17:19:11 +0000 UTC
 Prov Recipe ID: 591b348ed1db8f0014000024
 CA Certificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0t... (Use --fullca for certificate)
    Web UI Link: https://app.compose.io/compose-3/deployments/newexampledb
         Health:
            SSH:
          Admin:
       SSHAdmin:
    CLI Connect: [mongo --ssl --sslAllowInvalidCertificates sl-eu-lon-2-portal.0.dblayer.com:17783/admin -u admin -p VUHUEQFAGFQUUJWW]
 Direct Connect: [mongodb://admin:VUHUEQFAGFQUUJWW@sl-eu-lon-2-portal.0.dblayer.com:17783,sl-eu-lon-2-portal.5.dblayer.com:17783/admin?ssl=true]

About ...

... a database

You can find out more about a database with the bach scale command. Give it a deployment id and it will tell you how many units of resource sare allocated to the deployment. If you want to increase that allocation, bach scale set will let you increase, or decrease, the resource and the cost of the database deployment.

The bach alerts command will return the current alerts for the deployment. These are the alerts displayed in the Compose console's Overview and reflect issues that may, or may not, be impacting upon the deployments performance.

.... a user

The bach account command looks up the account which the Compose API Token was created for. The account can have multiple users associated with it and to get that information, the bach user command will give the user id back.

Deleting a database

The last command we'll touch on is bach deprovision which takes a deployment id as a parameter and, as it says, deprovisions it. Actually, it starts the deprovisioning and returns a recipe id. You can watch that recipe using bach watch as it tears down the database cluster.

Flagging things up

There's a couple of flags which may be useful to script developers. If we add the --json flag to any command, it'll attempt to return JSON data, post processed by Bach. So, if we add that flag to the bach backups list command we used earlier:

$ bach backups list 591af872cb404d0015001c9b --json
[
 {
  "id": "591b03b6c381128381000059",
  "deployment_id": "591af872cb404d0015001c9b",
  "name": "exampledb_2017-05-16_13-50-46_utc_on_demand",
  "type": "on_demand",
  "status": "complete",
  "download_link": ""
 }
]

There's also a --raw command if you want to see exactly what the API is returning:

$ bach backups list 591af872cb404d0015001c9b --raw
{"_embedded":{"backups":[{"id":"591b03b6c381128381000059","deployment_id":"591af872cb404d0015001c9b","name":"exampledb_2017-05-16_13-50-46_utc_on_demand","type":"on_demand","status":"complete","created_at":"2017-05-16T13:50:46.198Z","_links":{"self":{"href":"deployments/591af872cb404d0015001c9b/backups/591b03b6c381128381000059{?embed}","templated":true}}}]}}

Finally

The bach utility will always be an open-sourced work-in-progress. You can use the binary versions of it as is to ease your command-line interactions with Compose's API. The source code is available under an Apache 2.0 license too, so you can create your own enhancements - we welcome pull requests. You can find the Github repository for Bach at compose/bach. Do let us know how you get on as we build out Bach to make your command line Compose experience easier than ever.


If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at articles@compose.com. We're happy to hear from you.

Dj Walker-Morgan
Dj Walker-Morgan is Compose's resident Content Curator, and has been both a developer and writer since Apples came in II flavors and Commodores had Pets. Love this article? Head over to Dj Walker-Morgan’s author page and keep reading.