Connecting to RethinkDB with Elixir

Published

When you have a database that's as interesting as RethinkDB hosted on Compose, it's not going to be a surprise to find people using cutting edge languages to connect to it. With that in mind, we're going to look at how to connect to Compose RethinkDB using Elixir and pick out some of the pitfalls. We're going to be using the rethinkdb-elixir driver to connect to the database; it's a pure Elixir driver.

Preparation

We'll assume that you've got Elixir installed and the mix and iex commands available - we'll create ourselves a mix project:

$ mix new elxrdb
mix new elxrdb  
* creating README.md
* creating .gitignore
* creating mix.exs
* creating config
* creating config/config.exs
* creating lib
* creating lib/elxrdb.ex
* creating test
* creating test/test_helper.exs
* creating test/elxrdb_test.exs

Your Mix project was created successfully.  
You can use "mix" to compile it, test it, and more:

    cd elxrdb
    mix test

Run "mix help" for more commands.  
$ cd elxrdb
$ mix test
Compiling 1 file (.ex)  
Generated elxrdb app  
.

Finished in 0.02 seconds  
1 test, 0 failures

Randomized with seed 567600

And that's our example project created. Now we need to add in the rethinkdb library. We do that by opening up mix.exs in our preferred editor and adding {:rethinkdb, "~> 0.4.0"} within the defp deps do section like so:

With that in place, we can run mix deps.get to get the required packages:

$ mix deps.get
Running dependency resolution  
* Getting rethinkdb (Hex package)
  Checking package (https://repo.hex.pm/tarballs/rethinkdb-0.4.0.tar)
  Using locally cached package
* Getting connection (Hex package)
  Checking package (https://repo.hex.pm/tarballs/connection-1.0.3.tar)
  Using locally cached package
* Getting poison (Hex package)
  Checking package (https://repo.hex.pm/tarballs/poison-2.2.0.tar)
  Using locally cached package
$

Certificates Please

Before we can connect, we need to get the SSL Public Certificate from the Compose RethinkDB deployment. You'll find it on the Overview page of your Compose. It's normally hidden, so go down the page to where it says SSL Certificate (Self-Signed), click the Show button and, from the text area that appears, cut and paste the entire contents into a local file using an editor. Save it as compose.cert. We'll use it in a moment.

While we are on this page we can gather the other information we'll need to make the connection. Look up to the Connection Info panel. It'll look like this:

Take the connection string, and make a note of the hostname and port. We've marked them above so you can find them easily. The other thing you'll need is the authkey. Before RethinkDB 2.3, the authkey was a kind of password for the RethinkDB cluster. Since RethinkDB 2.3, that same key is now the password for the admin user. Because it has two uses, we've labelled it Authentication Credential in the Console. The current version of Elixir-RethinkDB hasn't migrated to RethinkDB 2.3 yet so we'll be using it as an authkey. Click the Show button to reveal it and make a note of it.

Getting Interactive

We're going to test everything is in place by running the Elixir interactive shell. Run iex -S mix and you'll see something like this:

iex -S mix  
Erlang/OTP 19 [erts-8.0.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

==> connection
Compiling 1 file (.ex)  
Generated connection app  
==> poison
Compiling 4 files (.ex)  
Generated poison app  
==> rethinkdb
Compiling 12 files (.ex)  
Generated rethinkdb app  
==> elxrdb
Compiling 1 file (.ex)  
Generated elxrdb app  
Interactive Elixir (1.3.2) - press Ctrl+C to exit (type h() ENTER for help)  
iex(1)>

Now we can import the RethinkDB library with import RethinkDB:

iex(1)> import RethinkDB  
RethinkDB  
iex(2)> { :ok, conn } = RethinkDB.Connection.start_link([ host: "sl-eu-lon-2-portal.2.dblayer.com", port: 10354, auth_key: "password", ssl: %{ ca_certs: ["./compose.cert" ] }, db: "spystuff"])  
{:ok, #PID<0.570.0>}
iex(3)>

There's a lot there so let's break it down. We're invoking the start_link function of RethinkDB.Connection. It takes a map of options. First up are the details we got from the connection string; host: is the host name to connect to, this can be a string or a char list, and port: is the numeric port number. Then comes the authorization key we copied, passed as auth_key:.

Next come the SSL details, passed over as it's own map. That only uses one entry at the moment, ca_certs: which should be set to be a list of file paths where certificates can be found. We've only got the one we created earlier. There's one last parameter we can pass, db:, which sets the name of the default database within RethinkDB we'll be using. We've set that to our spystuff database (which we used in our RethinkDB Joinery article).

This call returns a process for the connection to RethinkDB. Now we can use it:

iex(3)> results=RethinkDB.Query.table("agents") |> RethinkDB.run(conn)  
%RethinkDB.Collection{data: [%{"id" => "03c90deb-4c85-49b9-bbf9-0c6945bf04bb",
    "name" => "Illya Kuryakin",
    "org_id" => "7f486769-a43e-44d9-bf85-74e8859c6b7e", "skill" => ["combat"]},
  %{"id" => "1934a1f7-0cac-4ccd-a8d8-b4d641313935", "name" => "Chuck Bartowski",
    "org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
    "skill" => ["investigation", "stealth"]},
    ...

This is already looking a little clumsy, but you can import the RethinkDB.Query module itself which then lets write more concise statements:

iex(4)> import RethinkDB.Query  
RethinkDB.Query  
iex(5)> results=table("agents") |> eq_join("org_id",table("orgs")) |> zip() |> run(conn)  
%RethinkDB.Collection{data: [%{"alignment" => %{"country" => "USA",
      "side" => "west"}, "id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
    "name" => "Chuck Bartowski", "org" => "CIA",
    "org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
    "skill" => ["investigation", "stealth"]},
  %{"alignment" => %{"country" => "USA", "side" => "west"},
    "id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4", "name " => "Jason Bourne",
    "org" => "CIA", "org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
    "skill" => ["combat", "assassination"]},
    ....

With the namespace clutter gone, we can query, eq_join and zip together two tables with ease. Note we also just used run(conn) to execute the query.

We're up and running with the interactive Elixir shell and it's a great place to start exploring the RethinkDB-Elixir package. But we want to make more of an application so let's quit iex and do that.

A Simple Application

First stop is to add a supervised connection from Elixir's OTP to RethinkDB. The OTP, Open Telecom Platform, is a group of libraries that ship with Elixir and offer a framework and tools for managing the fault-tolerant, robust applications that are the trademark of the Erlang family.

We could get by with using the commands we used above to make queries, but that's not the Elixir (or Erlang) way. For practical applications, the OTP platform has the concepts of supervisors and workers and managed startups. So as part of a managed startup, we're going to create a supervisor which has one worker whose task is to manage a connection to RethinkDB. You can read more about the supervisor and worker concept in the Elixir documentation.

So, let's edit lib/elxrdb.ex so it looks like this:

defmodule Elxrdb do  
  def start(_type, _args) do
    import Supervisor.Spec
    children = [
      worker(Elxrdb.Connection, [[ host: "sl-eu-lon-2-portal.2.dblayer.com", port: 10354, auth_key: "password", ssl: %{ ca_certs: ["./compose.cert" ] }, db: "spystuff"]])
  ]
    Supervisor.start_link(children, strategy: :one_for_one, name: Elxrdb.Supervisor)
  end
end

defmodule Elxrdb.Connection do  
    use RethinkDB.Connection
end

This defines the startup for our application. It creates children for a supervisor, of which there's only one, a worker, named Elxrdb.Connection. That worker will be presented with one argument when invoked, a list of options which should be familiar as they are the options we used to create a connection interactively. With the children defined, the start_link function is called for Supervisor, which will, in turn, invoke start_link on all the children. And how does that connect up with the RethinkDB libraries? The last module definition defines Elxrdb.Connection as using RethinkDB.Connection to make that link.

To make that code run at startup, we need to modify the mix.exs file, this time changing the application definition from this:

def application do  
    [applications: [:logger]]
end

to this:

def application do  
    [applications: [:logger],
    mod: {Elxrdb, []}]
  end

This will now wake up our connection at startup. Now we can make a module that uses the database connection. Create lib/database.ex and add this to it:

defmodule Database do  
    import RethinkDB.Query, only: [table_create: 1, table: 1, insert: 2]

    def create_table(table_name) do
        table_create(table_name) |> Elxrdb.Connection.run
    end

    def create_entry(table_name, entry) do
        table(table_name) |> insert(%{title: entry}) |> Elxrdb.Connection.run
    end
end

The module imports specific parts of RethinkDB.Query which it then uses to form a "create table" function and a "create entry" function. Note how it refers to the Elxrdb.Connection to run the queries.

With this in place, we can test things out in the interactive shell, running iex -S mix:

iex -S mix  
Erlang/OTP 19 [erts-8.0.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Compiling 1 file (.ex)  
Interactive Elixir (1.3.2) - press Ctrl+C to exit (type h() ENTER for help)  
iex(1)>

The connection to RethinkDB is already up and running and we can invoke the Database module to create a table and add an entry:

iex(1)> Database.create_table("secretmessages")  
%RethinkDB.Record{data: %{"config_changes" => [%{"new_val" => %{"db" => "spystuff",
        "durability" => "hard", "id" => "90be664d-928d-4193-bdee-1e8f01732730",
        "indexes" => [], "name" => "secretmessages", "primary_key" => "id",
        "shards" => [%{"nonvoting_replicas" => [],
           "primary_replica" => "rethink328_sl_eu_lon_2_data_3_dblayer_com",
           "replicas" => ["rethink328_sl_eu_lon_2_data_3_dblayer_com"]}],
        "write_acks" => "majority"}, "old_val" => nil}], "tables_created" => 1},
 profile: nil}
iex(2)> Database.create_entry("secretmessages","Ixnay On The Messagay")  
%RethinkDB.Record{data: %{"deleted" => 0, "errors" => 0,
   "generated_keys" => ["08faf062-bf86-416c-b9ef-e2c252de0ffb"],
   "inserted" => 1, "replaced" => 0, "skipped" => 0, "unchanged" => 0},
 profile: nil}
iex(3)>

And that connection is available for our own ad-hoc queries too:

iex(3)> import RethinkDB  
RethinkDB  
iex(4)> import RethinkDB.Query  
RethinkDB.Query  
iex(5)> table("agents") |> run(Elxrdb.Connection)  
%RethinkDB.Collection{data: [%{"id" => "03c90deb-4c85-49b9-bbf9-0c6945bf04bb",
    "name" => "Illya Kuryakin",
    "org_id" => "7f486769-a43e-44d9-bf85-74e8859c6b7e", "skill" => ["combat"]},
  %{"id" => "1934a1f7-0cac-4ccd-a8d8-b4d641313935", "name" => "Chuck Bartowski",
    "org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
    "skill" => ["investigation", "stealth"]},
 ...

Wrapping up

The RethinkDB Elixir driver is an idiomatic implementation of ReQLs format which makes it fairly simple to translate ReQL command chains from other languages to Elixir piping. Elixir's interactive shell also makes it easy to explore the driver, and RethinkDBs capabilities. Hopefully, this tutorial will get you up and running with Compose's RethinkDB deployments more rapidly. We're looking forward to the next release of the driver which will hopefully support RethinkDB 2.3's authentication scheme. Until then though, enjoy your Elixir with Compose RethinkDB and the RethinkDB-Elixir driver.

Image by Drew Collins
Dj Walker-Morgan
Dj Walker-Morgan is Compose's resident Content Curator, and has been both a developer and writer since Apples came in II flavors and Commodores had Pets. Love this article? Head over to Dj Walker-Morgan’s author page and keep reading.