Redis PubSub, Node, and Socket.io

Published

Sockets are the high power pipeline of the realtime web and in this article we'll show how a minimal amount of code can bring database data to life in a web browser.

With the rise of bots and the chat based tools such as Slack and Messenger, users today have come to expect much more immediate interactions from their applications. One of the tools that most front end developers should have in their toolbox today is socket based communication. With a socket based solution it is easy to deliver realtime updating like leaderboards, stock quotes, tweets or any other streaming style of data to both mobile and web applications.

Here we will look at using just such a set of tools with NodeJS and Socket.io on both the server and in the browser. And we will complement them with a Redis PubSub implementation to model interacting with backend services and Smoothie.js to finish off the front end with a visualization. We'll use tweets as an example but it is easy to substitute any kind of realtime data you may have available.

NodeJS and Socket.io

We need three things on the server side. First something to serve a web page since that in essence is our front end application. ExpressJS works just fine:

var express = require('express');  
var app = express();  
var http = require('http').Server(app);

app.use('/', express.static('www'));

http.listen(8000, function(){  
  console.log('listening on *:8000');
});

Above we setup Node as an HTTP server to deliver our web page (application) which is just some static assets in the www directory.

Second we create our server's socket infrastructure with Socket.io:

var io = require('socket.io')(http);

Seriously, that's it for setting it up. We haven't sent any messages yet, nor received any, but the infrastructure is now in place. And what is most interesting is that this will work over most current infrastructure because it starts with long-polling and then upgrades the connection to an actual websocket. So, you can use the socket model even without sockets currently. See engine-io for details.

Third, we'll include a Redis subscription and wire up broadcasting an actual Socket.io message:

var redis = require('redis');  
var url = config.get('redis.url');  
var client1 = redis.createClient(url);  
var client2 = redis.createClient(url);

client1.on('message', function(chan, msg) {  
  client2.hgetall(msg, function(err, res) {
      res.key = msg;
      io.sockets.emit(res);
  });
});

client1.subscribe('yourChannelName');

We use two Redis connections. client1 handles the PubSub subscription while client2 actually gets the hash for the key that came through the subscription (it is be possible to remove the second connection and push all of the data through the PubSub channel too). Then with io.sockets.emit(res); we broadcast all of the data to any connected clients. We've left out the Redis publish side of above but it really isn't any more complicated than reversing what we've shown:

client.publish("yourChannelName", msg);  

As you can see the simplicity of this highlights how effective Node is as an event based networking tool. Next we'll move on to the client side which listens for the broadcast.

Web Browser and Socket.io

As you might have guessed the browser side of Socket.io is pretty easy too. So, with the assumption that an html page has been delivered to your browser via Express and your Node server then the following sock.on() will be called every time a broadcast message is emitted from your server. The beauty here is that the Socket.io library defaults to contacting the same server which delivered the page.

<script src="https://cdn.socket.io/socket.io-1.4.5.js"></script>  
<script>  
  var socket = io();

  sock.on('twits', function(msg) {
    //Do something with message here. 
  });

</script>

That little bit of script is perfect for handing a continuous stream of events off to a realtime charting tool. While there are lots of JavaScript charting libraries, one of the easiest for this style of data is Smoothie.js.

To use it set up a <canvas id='twits'> tag in the body of an html page and then you can attach the chart and stream the data to it.

The JavaScript to wire all of the charting up, attach it to the canvas, and stream follows:

function createGraphOnPageLoad() {  
  var sock = io();

  var smoothie = new SmoothieChart();
  smoothie.streamTo(document.getElementById('twits'));

  var redLine = new TimeSeries();
  var blueLine = new TimeSeries();

  smoothie.addTimeSeries(redLine,{ strokeStyle:'rgb(255, 0, 0)', lineWidth:3 } );
  smoothie.addTimeSeries(blueLine, { strokeStyle:'rgb(0, 0, 255)', lineWidth:3 });

  sock.on('twits', function(msg) {
    var at = new Date().getTime();
    var reach = msg.reach * 1;
    if(msg.category == "Red") {
      redLine.append(at, reach);
    } else {
      blueLine.append(at, reach);
    }
  });
}

The above function should be called after the page loads which ensures that the canvas element is already created.

It creates the socket, creates the chart and wires it to the canvas.

Then it creates two timeSeries. The messages actually represent tweets and the reach is the number of people who receive the tweet. The Redis PubSub actually transports both red and blue categorized tweets. The timelines represent how many followers could see the tweet on the y-axis and time on the x-axis. Red and blue are categories of twitter searches for comparison. It adds them to the chart at which point it waits for the events which are actually tweets and then it appends them. On each append the chart is updated with the reach metric and inserted at the current time. The web page output looks like this:

Reach

While it is a simple charting solution it does a good job of showing the value of the full chain of soft realtime data via Node, Redis, and Socket.io.

To view the code example on github go here.


If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at articles@compose.com. We're happy to hear from you.

Image by William Iven

Conquer the Data Layer

Spend your time developing apps, not managing databases.