Connecting to RethinkDB with Elixir
PublishedWhen you have a database that's as interesting as RethinkDB hosted on Compose, it's not going to be a surprise to find people using cutting edge languages to connect to it. With that in mind, we're going to look at how to connect to Compose RethinkDB using Elixir and pick out some of the pitfalls. We're going to be using the rethinkdb-elixir driver to connect to the database; it's a pure Elixir driver.
Preparation
We'll assume that you've got Elixir installed and the mix
and iex
commands available - we'll create ourselves a mix project:
$ mix new elxrdb
mix new elxrdb
* creating README.md
* creating .gitignore
* creating mix.exs
* creating config
* creating config/config.exs
* creating lib
* creating lib/elxrdb.ex
* creating test
* creating test/test_helper.exs
* creating test/elxrdb_test.exs
Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:
cd elxrdb
mix test
Run "mix help" for more commands.
$ cd elxrdb
$ mix test
Compiling 1 file (.ex)
Generated elxrdb app
.
Finished in 0.02 seconds
1 test, 0 failures
Randomized with seed 567600
And that's our example project created. Now we need to add in the rethinkdb library. We do that by opening up mix.exs
in our preferred editor and adding {:rethinkdb, "~> 0.4.0"}
within the defp deps do
section like so:
With that in place, we can run mix deps.get
to get the required packages:
$ mix deps.get
Running dependency resolution
* Getting rethinkdb (Hex package)
Checking package (https://repo.hex.pm/tarballs/rethinkdb-0.4.0.tar)
Using locally cached package
* Getting connection (Hex package)
Checking package (https://repo.hex.pm/tarballs/connection-1.0.3.tar)
Using locally cached package
* Getting poison (Hex package)
Checking package (https://repo.hex.pm/tarballs/poison-2.2.0.tar)
Using locally cached package
$
Certificates Please
Before we can connect, we need to get the SSL Public Certificate from the Compose RethinkDB deployment. You'll find it on the Overview page of your Compose. It's normally hidden, so go down the page to where it says SSL Certificate (Self-Signed), click the Show button and, from the text area that appears, cut and paste the entire contents into a local file using an editor. Save it as compose.cert
. We'll use it in a moment.
While we are on this page we can gather the other information we'll need to make the connection. Look up to the Connection Info panel. It'll look like this:
Take the connection string, and make a note of the hostname and port. We've marked them above so you can find them easily. The other thing you'll need is the authkey. Before RethinkDB 2.3, the authkey was a kind of password for the RethinkDB cluster. Since RethinkDB 2.3, that same key is now the password for the admin user. Because it has two uses, we've labelled it Authentication Credential in the Console. The current version of Elixir-RethinkDB hasn't migrated to RethinkDB 2.3 yet so we'll be using it as an authkey. Click the Show button to reveal it and make a note of it.
Getting Interactive
We're going to test everything is in place by running the Elixir interactive shell. Run iex -S mix
and you'll see something like this:
iex -S mix
Erlang/OTP 19 [erts-8.0.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
==> connection
Compiling 1 file (.ex)
Generated connection app
==> poison
Compiling 4 files (.ex)
Generated poison app
==> rethinkdb
Compiling 12 files (.ex)
Generated rethinkdb app
==> elxrdb
Compiling 1 file (.ex)
Generated elxrdb app
Interactive Elixir (1.3.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
Now we can import the RethinkDB library with import RethinkDB
:
iex(1)> import RethinkDB
RethinkDB
iex(2)> { :ok, conn } = RethinkDB.Connection.start_link([ host: "sl-eu-lon-2-portal.2.dblayer.com", port: 10354, auth_key: "password", ssl: %{ ca_certs: ["./compose.cert" ] }, db: "spystuff"])
{:ok, #PID<0.570.0>}
iex(3)>
There's a lot there so let's break it down. We're invoking the start_link
function of RethinkDB.Connection. It takes a map of options. First up are the details we got from the connection string; host:
is the host name to connect to, this can be a string or a char list, and port:
is the numeric port number. Then comes the authorization key we copied, passed as auth_key:
.
Next come the SSL details, passed over as it's own map. That only uses one entry at the moment, ca_certs:
which should be set to be a list of file paths where certificates can be found. We've only got the one we created earlier. There's one last parameter we can pass, db:
, which sets the name of the default database within RethinkDB we'll be using. We've set that to our spystuff
database (which we used in our RethinkDB Joinery article).
This call returns a process for the connection to RethinkDB. Now we can use it:
iex(3)> results=RethinkDB.Query.table("agents") |> RethinkDB.run(conn)
%RethinkDB.Collection{data: [%{"id" => "03c90deb-4c85-49b9-bbf9-0c6945bf04bb",
"name" => "Illya Kuryakin",
"org_id" => "7f486769-a43e-44d9-bf85-74e8859c6b7e", "skill" => ["combat"]},
%{"id" => "1934a1f7-0cac-4ccd-a8d8-b4d641313935", "name" => "Chuck Bartowski",
"org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
"skill" => ["investigation", "stealth"]},
...
This is already looking a little clumsy, but you can import the RethinkDB.Query module itself which then lets write more concise statements:
iex(4)> import RethinkDB.Query
RethinkDB.Query
iex(5)> results=table("agents") |> eq_join("org_id",table("orgs")) |> zip() |> run(conn)
%RethinkDB.Collection{data: [%{"alignment" => %{"country" => "USA",
"side" => "west"}, "id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
"name" => "Chuck Bartowski", "org" => "CIA",
"org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
"skill" => ["investigation", "stealth"]},
%{"alignment" => %{"country" => "USA", "side" => "west"},
"id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4", "name " => "Jason Bourne",
"org" => "CIA", "org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
"skill" => ["combat", "assassination"]},
....
With the namespace clutter gone, we can query, eq_join and zip together two tables with ease. Note we also just used run(conn)
to execute the query.
We're up and running with the interactive Elixir shell and it's a great place to start exploring the RethinkDB-Elixir package. But we want to make more of an application so let's quit iex and do that.
A Simple Application
First stop is to add a supervised connection from Elixir's OTP to RethinkDB. The OTP, Open Telecom Platform, is a group of libraries that ship with Elixir and offer a framework and tools for managing the fault-tolerant, robust applications that are the trademark of the Erlang family.
We could get by with using the commands we used above to make queries, but that's not the Elixir (or Erlang) way. For practical applications, the OTP platform has the concepts of supervisors and workers and managed startups. So as part of a managed startup, we're going to create a supervisor which has one worker whose task is to manage a connection to RethinkDB. You can read more about the supervisor and worker concept in the Elixir documentation.
So, let's edit lib/elxrdb.ex
so it looks like this:
defmodule Elxrdb do
def start(_type, _args) do
import Supervisor.Spec
children = [
worker(Elxrdb.Connection, [[ host: "sl-eu-lon-2-portal.2.dblayer.com", port: 10354, auth_key: "password", ssl: %{ ca_certs: ["./compose.cert" ] }, db: "spystuff"]])
]
Supervisor.start_link(children, strategy: :one_for_one, name: Elxrdb.Supervisor)
end
end
defmodule Elxrdb.Connection do
use RethinkDB.Connection
end
This defines the startup for our application. It creates children for a supervisor, of which there's only one, a worker, named Elxrdb.Connection
. That worker will be presented with one argument when invoked, a list of options which should be familiar as they are the options we used to create a connection interactively. With the children defined, the start_link
function is called for Supervisor, which will, in turn, invoke start_link
on all the children. And how does that connect up with the RethinkDB libraries? The last module definition defines Elxrdb.Connection
as using RethinkDB.Connection
to make that link.
To make that code run at startup, we need to modify the mix.exs
file, this time changing the application definition from this:
def application do
[applications: [:logger]]
end
to this:
def application do
[applications: [:logger],
mod: {Elxrdb, []}]
end
This will now wake up our connection at startup. Now we can make a module that uses the database connection. Create lib/database.ex
and add this to it:
defmodule Database do
import RethinkDB.Query, only: [table_create: 1, table: 1, insert: 2]
def create_table(table_name) do
table_create(table_name) |> Elxrdb.Connection.run
end
def create_entry(table_name, entry) do
table(table_name) |> insert(%{title: entry}) |> Elxrdb.Connection.run
end
end
The module imports specific parts of RethinkDB.Query which it then uses to form a "create table" function and a "create entry" function. Note how it refers to the Elxrdb.Connection
to run the queries.
With this in place, we can test things out in the interactive shell, running iex -S mix
:
iex -S mix
Erlang/OTP 19 [erts-8.0.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Compiling 1 file (.ex)
Interactive Elixir (1.3.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
The connection to RethinkDB is already up and running and we can invoke the Database module to create a table and add an entry:
iex(1)> Database.create_table("secretmessages")
%RethinkDB.Record{data: %{"config_changes" => [%{"new_val" => %{"db" => "spystuff",
"durability" => "hard", "id" => "90be664d-928d-4193-bdee-1e8f01732730",
"indexes" => [], "name" => "secretmessages", "primary_key" => "id",
"shards" => [%{"nonvoting_replicas" => [],
"primary_replica" => "rethink328_sl_eu_lon_2_data_3_dblayer_com",
"replicas" => ["rethink328_sl_eu_lon_2_data_3_dblayer_com"]}],
"write_acks" => "majority"}, "old_val" => nil}], "tables_created" => 1},
profile: nil}
iex(2)> Database.create_entry("secretmessages","Ixnay On The Messagay")
%RethinkDB.Record{data: %{"deleted" => 0, "errors" => 0,
"generated_keys" => ["08faf062-bf86-416c-b9ef-e2c252de0ffb"],
"inserted" => 1, "replaced" => 0, "skipped" => 0, "unchanged" => 0},
profile: nil}
iex(3)>
And that connection is available for our own ad-hoc queries too:
iex(3)> import RethinkDB
RethinkDB
iex(4)> import RethinkDB.Query
RethinkDB.Query
iex(5)> table("agents") |> run(Elxrdb.Connection)
%RethinkDB.Collection{data: [%{"id" => "03c90deb-4c85-49b9-bbf9-0c6945bf04bb",
"name" => "Illya Kuryakin",
"org_id" => "7f486769-a43e-44d9-bf85-74e8859c6b7e", "skill" => ["combat"]},
%{"id" => "1934a1f7-0cac-4ccd-a8d8-b4d641313935", "name" => "Chuck Bartowski",
"org_id" => "938201f0-6a24-4f0a-91ee-cc1751df23a4",
"skill" => ["investigation", "stealth"]},
...
Wrapping up
The RethinkDB Elixir driver is an idiomatic implementation of ReQLs format which makes it fairly simple to translate ReQL command chains from other languages to Elixir piping. Elixir's interactive shell also makes it easy to explore the driver, and RethinkDBs capabilities. Hopefully, this tutorial will get you up and running with Compose's RethinkDB deployments more rapidly. We're looking forward to the next release of the driver which will hopefully support RethinkDB 2.3's authentication scheme. Until then though, enjoy your Elixir with Compose RethinkDB and the RethinkDB-Elixir driver.