From Node.js to Redis and RabbitMQ: The Compose Grand Tour
PublishedIn this stage of the Compose Grand Tour for Node.js, we'll be looking at connecting to Redis with both popular drivers and plugging into RabbitMQ.
The Compose Grand Tour is a series of articles and examples which show how to connect to the nine databases of Compose with different languages. In previous tours, we've covered Go and Python and this is the second stage of the Node.js/JavaScript tour.
In the first stage of the Node.js tour we covered MongoDB, Elasticsearch, and PostgreSQL and talked about the common example code which nearly all the examples use. Now it's time for Redis and RabbitMQ. First up, Redis.
Redis
For Redis, we've actually made two examples. One for node-redis and one for ioredis, both popular client packages for Redis. The thing is, we've been expecting the two projects to merge for a while but they haven't visibly done so. Node-redis is the older driver and its API is callback oriented, while the younger ioredis has native support for promises and callbacks. So, to make your life easier, we've done an example for each.
Redis with node-redis
The code for this example is in the example-redis directory of the Node Grand Tour Repository.
The Node-Redis Driver
This example is for the node-redis package, so we include that with const redis = require("redis");.
The Node-Redis Connection
As with all connections in the Grand Tour, we get our connection string from the environment variables. In this case, it is COMPOSE_REDIS_URL.
let connectionString = process.env.COMPOSE_REDIS_URL;
if (connectionString === undefined) {
console.error("Please set the COMPOSE_REDIS_URL environment variable");
process.exit(1);
}
We exit if that's not been set. Once we have it, we move on to the connection creation. The redis.connect() method takes a connection string or a connection string with a hash of options. Although it detects the use of the rediss:// protocol in the connection string to perform TLS/SSL encryption, it doesn't set the TLS handshake servername which we need set with Compose deployments. So the first thing we do is spot the rediss:// protocol at the start. If it is present, we create the client while passing it the tls.servername setting. The value for that setting comes from parsing the connection string with the URL package and extracting the hostname...
let client = null;
if (connectionString.startsWith("rediss://")) {
client = redis.createClient(connectionString, {
tls: { servername: new URL(connectionString).hostname }
});
}
Of course, if there's no TLS enabled, it's just a matter of creating the client:
else {
client = redis.createClient(connectionString);
}
The client is passive though and rather than keep any problem hidden till the first actual operation, we suggest that for long running connections you ping the server first. We do that in this example at the end before starting the server:
client.ping((err, reply) => {
if (err !== null) {
console.log(err);
process.exit(1);
}
app.listen(port, () => {
console.log("Server is listening on port " + port);
});
});
If something does go wrong, in the callback we report the error and exit. Otherwise, we start the web-server. Now on to the reading and writing.
The Node-Redis Read
With Redis, we keep our words in a hash; each word is the key, the definition is the value. To get all the values from a hash, Redis has the HGETALL command, and that maps to the hgetall function in node-redis. From other Node Grand Tour examples, you should know we are expected to return a promise in getWords which will resolve to our words and definitions. So, here we wrap client.hgetall() with a promise. In the callback for hgetall if there was an error, we reject the promise. If there wasn't we resolve with our results:
function getWords() {
return new Promise((resolve, reject) => {
client.hgetall("words", (err, resp) => {
if (err) {
reject(err);
} else {
resolve(resp);
}
});
});
}
The Node-Redis Write
A similar thing has to be performed for setting a word's definition in the hash. The Redis command HSET maps to node-redis's hset function. We get to wrap that in a promise that resolves with a string of "success".
function addWord(word, definition) {
return new Promise((resolve, reject) => {
client.hset("words", word, definition, (error, result) => {
if (error) {
reject(error);
} else {
resolve("success");
}
});
});
}
And that's it for Redis with node-redis. Now let's look at the same with ioredis.
Redis with ioredis
The code for this example is in the example-redis-ioredis directory of the Node Grand Tour Repository.
The ioredis Driver
This example is for the ioredis package. We will include that with const Redis = require("ioredis");.
The ioredis Connection
The ioredis connect method supports being called with a whole variety of parameters. We're going to stick with the simple one, a connection string and a JSON object of options.
let url = new URL(connectionString);
let options = { };
if (url.protocol === "rediss:") {
options.tls = { servername: url.hostname };
}
let redis = new Redis(connectionString, options );
We already have the COMPOSE_REDIS_URL environment variable in as connectionString. This is converted into a URL object, url. We then create an empty options map.
There's only one option that might be set in those options. If the URL object has a protocol rediss: then we are connecting to a TLS protected Redis. We will need to set the tls.servername in the options map to the hostname of the server which we get from the url.
Then we can create the connection using the connection string and our freshly minted options.
As with node-redis, we ping the Redis server to make sure its there before starting the web server. This time, we do that ping() and a promise:
redis
.ping()
.then(() => {
app.listen(port, () => {
console.log("Server is listening on port " + port);
});
})
.catch(err => {
console.log(err);
});
The ioredis Read
As ioredis calls return Promises, the getWords() function simply has to call its hgetall function.
function getWords() {
return redis.hgetall("words");
}
The ioredis Write
Similarly to the read, the write simply calls the ioredis hset function:
function addWord(word, definition) {
return redis.hset("words", word, definition);
}
All of which shows how much you can simplify your code with Promises and make it much more predictable. So that's it for Redis and its two popular Node.js drivers. Next stop is RabbitMQ...
RabbitMQ
As with other Grand Tour languages, the RabbitMQ example is different because RabbitMQ is different. It involves pushing messages into an exchange and retrieving them from a queue.
The code for this example is in the example-rabbitmq directory of the Node Grand Tour Repository.
The RabbitMQ Driver
This example is for the amqplib package. It's also known as amqp.node. We will include that with const amqp = require("amqplib");. There are two versions of the API for this package, one with Promises and one with callbacks. We're going with the Promises one.
The RabbitMQ Connection
The connection process starts off with our environment variable COMPOSE_RABBITMQ_URL containing a connection string for a RabbitMQ deployment. We'll get that value, exiting if it isn't set, and then parse it as a URL.
let connectionString = process.env.COMPOSE_RABBITMQ_URL;
if (connectionString === undefined) {
console.error("Please set the COMPOSE_RABBITMQ_URL environment variable");
process.exit(1);
}
let parsedurl = url.parse(connectionString);
Now, there's some setup to do as we have some things to name.
let routingKey = "words";
let exchangeName = "grandtour";
let qName = "sample";
Now we're going to make a promise about connecting:
var open = amqp.connect(connectionString, { servername: parsedurl.hostname });
Now, when we use open we'll be referring to a promise that opens or has opened a connection. We're setting the optional servername field to the hostname in the connection string. Compose's RabbitMQ servers using SNI with TLS encryption to work out the right certificate for the server. Let's put this promise to work and set up our RabbitMQ server.
open
.then(conn => {
return conn.createChannel();
})
.then(ch => {
So we call up that open promise and that give us a connection. Using that, we create a channel to send commands over. That's another promise so when that resolves, we then have a channel to work with. Now, we can work with that channel.
We'll be using it to create an exchange called "grandtour". We'll then create a queue called "sample" and bind that to the "grandtour" exchange so that whenever a message arrives in the exchange with the routing key of "words" it lands in the "sample" queue.
return ch
.assertExchange(exchangeName, "direct", { durable: true })
.then(() => {
return ch.assertQueue(qName, { exclusive: false });
})
.then(q => {
return ch.bindQueue(q.queue, exchangeName, routingKey);
});
})
.catch(err => {
console.err(err);
process.exit(1);
});
Practically, we create the exchange with assertExchange. Then we create the queue with assertQueue and then we bind that queue with bindQueue. Each call returns a promise allowing them to be chained to be executed in sequence. We also catch any errors here and exit.
We are now ready to start the server to read and write messages.
The RabbitMQ Write
The first thing we do in out write is use the open connection to get us a channel.
function addMessage(message) {
return open
.then(conn => {
return conn.createChannel();
})
We then publish the message to our exchange "grandtour", with a routing key "words". Notice that we wrap the message in a buffer.
.then(ch => {
ch.publish(exchangeName, routingKey, new Buffer(message));
There's no promise or callback here. Our next task is to format up a message string, log it on the console and then return a Promise that'll resolve to that string.
let msgTxt = message + " : Message sent at " + new Date();
console.log(" [+] %s", msgTxt);
return new Promise(resolve => {
resolve(message);
});
});
}
And that's the message sent and that's writing to an exchange with Node.js. It gets a slightly more convoluted pulling that message out of the queue where it'll end up.
The RabbitMQ Read
The read starts off as the write does getting the connection and channel.
function getMessage() {
return open
.then(conn => {
return conn.createChannel();
})
.then(ch => {
Now, RabbitMQ has a number of ways to get messages from a queue but for our purposes, we're going to use the get function. When you pull a single message from the queue, you have to know if it's a message or there was nothing there to get. Enter the msgOrFalse return value when our get promise resolves. We create a whole new Promise to process this data.
In that promise, we start by setting our result to a "No messages" string. Then, if that msgOrFalse value isn't false then we create a new result message. If there was a message, we also acknowledge that we've processed it.
return ch.get(qName, {}).then(msgOrFalse => {
return new Promise(resolve => {
let result = "No messages in queue";
if (msgOrFalse !== false) {
result =
msgOrFalse.content.toString() +
" : Message received at " +
new Date();
ch.ack(msgOrFalse);
}
console.log(" [-] %s", result);
resolve(result);
});
});
});
}
Whatever the setting, we print the result to the console and then resolve the Promise with the same result.
And that's reading from a RabbitMQ queue with Node.js and the end of the RabbitMQ example.
Grand Tour Pit Stop
It's also the end of this part of the Node.js Grand Tour. In our next stage of the Grand Tour, we'll look at connecting to RethinkDB and Scylla, and end the journey with etcd and Compose for MySQL.
Read more articles about Compose databases - use our Curated Collections Guide for articles on each database type. If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at articles@compose.com. We're happy to hear from you.
attribution Peagreenbean