How to watch and wait with Compose's Bach and API

Published

We've just added a new feature to the Bach command line for the Compose API to make things easier to track to completion. Let us explain what --watch and --wait does with advice for command line users and API developers too.

If you've not met Bach yet, read all about the utility that lets your manage Compose from the command line in this article and download the latest version from the Bach GitHub repository.

The secret to great cooking is often just working through a recipe and waiting for things to be cooked just right. The same goes for the Compose API. It uses recipes too and you will need to wait till those recipes are completed to have your database deployment just right.

What's a recipe?

When you ask the Compose API to create a database deployment, backup on demand, scale up a database or decommission an existing database, what the Compose platform does is apply what it calls a recipe to the deployment. A recipe is a bundle of commands that get run against a deployment and that takes time. The Compose system checks back on the recipes over time to see when they are done and updates the status in the UI.

But when you are working with the Compose API, we prefer to return to as quickly as possible. That means returning as soon as we have an id for the recipe that's being used. This also happens with the Bach CLI. That can be a surprise if you don't know about it. More so if you expect that when you make an API call or run a command the work is all done by the time that they return.

Let's see this happening. We'll start an on-demand backup for a database. When you run a database backup you get back the recipe that's running the backup.

$ bach backups start 5927521082af7e0011000074
             ID: 592d4b38de099f00100001b0
       Template: Recipes::Deployment::Run
         Status: running
  Status Detail: Running usage on mongodb62.sl-eu-lon-2-portal.5.
     Account ID: 542da1926b9c7d465d00000a
  Deployment ID: 5927521082af7e0011000074
           Name: Backup
     Created At: 2017-05-30 10:36:40 +0000 UTC
     Updated At: 2017-05-30 10:36:41 +0000 UTC
  Child Recipes: 0
$ 

The command has returned, the backup is running but not completed. If we use the Bach watch command with the ID we got back we can see it working.

$ bach watch 592d4b38de099f00100001b0  
             ID: 592d4b38de099f00100001b0
       Template: Recipes::Deployment::Run
         Status: running
  Status Detail: Running backup_on_demand on sl-eu-lon-2-data.7.
     Account ID: 542da1926b9c7d465d00000a
  Deployment ID: 5927521082af7e0011000074
           Name: Backup
     Created At: 2017-05-30 10:36:40.686 +0000 UTC
     Updated At: 2017-05-30 10:36:41.145 +0000 UTC
  Child Recipes: 1

             ID: 592d4b38de099f00100001b0
       Template: Recipes::Deployment::Run
         Status: running
  Status Detail: Running backup_on_demand on sl-eu-lon-2-data.7.
     Account ID: 542da1926b9c7d465d00000a
  Deployment ID: 5927521082af7e0011000074
           Name: Backup
     Created At: 2017-05-30 10:36:40.686 +0000 UTC
     Updated At: 2017-05-30 10:36:41.145 +0000 UTC
  Child Recipes: 1

             ID: 592d4b38de099f00100001b0
       Template: Recipes::Deployment::Run
         Status: complete
  Status Detail: All operations have completed successfully!
     Account ID: 542da1926b9c7d465d00000a
  Deployment ID: 5927521082af7e0011000074
           Name: Backup
     Created At: 2017-05-30 10:36:40.686 +0000 UTC
     Updated At: 2017-05-30 10:37:08.648 +0000 UTC
  Child Recipes: 1
$

Every few seconds Bach polls the API for the latest status of the recipe until that status is complete. But that does involve you cutting and pasting the ID quickly into a new command and that is not something you want to do all the time.

Enter --watch (and --wait)

This is why we've added the --watch flag to Bach. With it, any command that returns a watchable recipe starts watching it automatically. Let's do that backup again, this time with the --watch flag.

$ bach backups start 5927521082af7e0011000074 --watch
             ID: 592d4d32de099f0018000172
       Template: Recipes::Deployment::Run
         Status: running
  Status Detail: Running usage on mongodb62.sl-eu-lon-2-portal.5.
     Account ID: 542da1926b9c7d465d00000a
  Deployment ID: 5927521082af7e0011000074
           Name: Backup
     Created At: 2017-05-30 10:45:06 +0000 UTC
     Updated At: 2017-05-30 10:45:06 +0000 UTC
  Child Recipes: 0

         Status: running
  Status Detail: Running backup_on_demand on sl-eu-lon-2-data.7.
     Updated At: 2017-05-30 10:45:06.62 +0000 UTC
....
         Status: complete
  Status Detail: All operations have completed successfully!
     Updated At: 2017-05-30 10:45:37.06 +0000 UTC
$

Now, where before it would have returned, the backup command starts monitoring the recipe. It does so in a more terse format than the bach watch command. It only outputs status, status message and updated at. It polls every five seconds, and if there's no change in those values, it just prints a progress full stop. If that's all a bit noisy for you, don't worry. There's also --wait which does exactly the same as --watch but produces no output beyond that which the unflagged command would produce. If you are scripting something, this may be the flag you find easiest to work with.

For API developers

As you can see from Bach's new flags, you'll probably be wanting to watch or wait for a recipe to complete for any endpoint that returns a recipe. API calls can do this indirectly - creating a deployment or restoring a backup as the provision_recipe_id field - or directly - creating a backup or deprovisioning a deployment.

Take that recipe id and poll the get recipe endpoint with that id. You'll get back a complete recipe which will tell you the status of that recipe. It'll be done when its state is either completed or failed. It's that simple; all you have to do is flow that process in with your code appropriately.

What next?

Recipes and how Compose uses them is one of the more interesting attributes of the Compose platform. Understanding that they need to run to completion on their own enables you to make more effective use of the API. It also keeps using Bach to work with that API an enjoyable process. Learn more about the API on its dedicated help site or, if you just want to use the command line - pick up the latest edition of Bach today.


If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at articles@compose.com. We're happy to hear from you.

Dj Walker-Morgan
Dj Walker-Morgan is Compose's resident Content Curator, and has been both a developer and writer since Apples came in II flavors and Commodores had Pets. Love this article? Head over to Dj Walker-Morgan’s author page to keep reading.

Conquer the Data Layer

Spend your time developing apps, not managing databases.