Tinkertank: MongoDB in Lights

TL;DR: How to put MongoDB statistics up in lights for ambient status displays.

In the previous MongoHQ Tinkertank article, we showed how to use Node.js, Mongo, Express and the Smoothie graphing library to create charts based on sampling the MongoDB oplog. This combination was put together in support of another project though, a plan to create physical statistics displays. To prove the concept we reached for an Arduino Uno board and one of Adafruit's NeoPixel shields. The shield clips to the top of the Arduino and lets the programmer drive 40 RGB LEDs, called NeoPixels by Adafruit, arranged in an eight by five grid. This will be enough for us to display three bar indicators of the statistics we harvested in the last article.

The Adafruit Neopixel shield
The Adafruit Neopixel shield in a moment of unblinkingness.

You will want to have the Arduino IDE already set up as we will have a program to load onto the device. Consult the Arduino Getting Started pages for instructions on how to configure it for your desktop platform. Assembling the hardware is simply plugging the NeoMatrix shield into the Arduino and connecting the Arduino to your system using an appropriate USB cable. We'll get back to programming the Arduino in a moment, but first we want to modify the Node.js software to drive our hardware.

Enhancing the Node

To connect our system we'll be using a serial connection over that USB connection and usefully, there's already a serial port package for Node.js called SerialPort. This lets us asynchronously handle the serial port. For this use, we'll just be writing to it but it is easy to imagine reading from the port to get information from sensors attached to the Arduino like light levels or temperature as part of an Internet of Things. Taking the lights-web.js application from the previous article, we will now perform some simple modifications.

First, after the initial "requires", we need to include the serialport package and configure it:

var SerialPort=require("serialport").SerialPort;  
var serialport=new SerialPort("/dev/tty.usbmodem8011", { baudrate: 9600 })  

You'll have to change this so that it addresses the correct device on your system. If you've configured your Arduino IDE correctly, you should find that it is displaying the serial port name in the bottom right of the IDE. Next we need to make the code write the new values to the serial port. In the sampler function, just after the io.sockets.emit(... line add:

serialport.write(insertCount+" "+updateCount+" "+deleteCount+"n", function(err,results) {  
  if(err) { console.log('err ' + err); }
})

And finally, we just need to make sure that serial port has opened before we begin so switch the final sampler(); line for

serialport.on("open", function() {  
  sampler();
});

Lighting up the Arduino

And that should be all the changes done. If you run this now, it'll crash as there's nothing listening on the serial port. Let's get the Arduino code up and running. For this code we've used the Adafruit NeoPixel, NeoMatrix and GFX libraries and they need to be installed in the IDE before you can progress. There's an UberGuide at Adafruit on how to drive the various RGB LED displays that are available. Many of them are strips of LEDs, so the NeoPixel library has an API which addresses them as a single row. The NeoMatrix library then does the mapping work needed to address them as an n-by-n array and the GFX library builds on top of that for graphics primitives. There are instructions for installing NeoPixel, NeoMatrix and GFX libraries in the Adafruit guide. If you have time, read the rest of the guide as it will describe how the RGB LEDs work and how strips and arrays of them can be wired to work together for even bigger displays.

We've put the sketch for the Arduino code in the Github repository with the Node.js code, but we'll step through it here to give you a feel. Probably the most obscure part of the code is the opening stanzas:

#include <Adafruit_GFX.h>
#include <Adafruit_NeoMatrix.h>
#include <Adafruit_NeoPixel.h>
#ifndef PSTR
 #define PSTR // Make Arduino Due happy
#endif

#define PIN 6

Adafruit_NeoMatrix matrix = Adafruit_NeoMatrix(5, 8, PIN,  
  NEO_MATRIX_TOP     + NEO_MATRIX_RIGHT +
  NEO_MATRIX_COLUMNS + NEO_MATRIX_PROGRESSIVE,
  NEO_GRB            + NEO_KHZ800);

These include the Adafruit libraries and then configure the matrix as a 5 by 8 array on pin 6 of the Arduino. Now we can begin with setting up some variables and preparing to run...

int insertops[]= { 0, 0, 0, 0, 0 };  
int updateops[]= { 0, 0, 0, 0, 0 };  
int deleteops[]= { 0, 0, 0, 0, 0 };

bool newdata=true;

void setup() {  
  matrix.begin();
  matrix.setBrightness(64);
  Serial.begin(9600);
}

The three arrays are going to be used to keep recent values which will be used to 'scale' our charts. The newdata flag will signal when we need to redraw later. In Arduino programming the setup() function is always called first. Here, we intialize the matrix and set its brightness to 64, a quarter of its maximum because these LEDs are bright and we assume you like your retinas. We also initialise the serial port to operate at 9600 baud, matching what we set in the Node.js code.

Arduino programs spend their run time forever tearing around a loop() function, which is the next thing we'll define.

  void loop() {
    if(newdata) {
drawVals();  
newdata=false;  
    }

If there is new data to be drawn, we draw it here. Otherwise we go on to read data from the serial port...

if(Serial.available() >0) {  
  int newiop=Serial.parseInt();
  int newuop=Serial.parseInt();
  int newdop=Serial.parseInt();

If there is some serial communications, we try to read three integers from it. If that succeeds we then look for the end of line and if we get that we do some processing, rotating the arrays we defined earlier, inserting our new values and setting the newdata flag so it'll redraw everything next time round.

    if(Serial.read() == 'n') {
int t=0;  
for(t=matrix.width()-1;t>0;t--) {  
        insertops[t]=insertops[t-1];
        updateops[t]=updateops[t-1];
        deleteops[t]=deleteops[t-1];
}
insertops[t]=newiop;  
updateops[t]=newuop;  
deleteops[t]=newdop;  
newdata=true;  
    }
  }
  delay(500);
}

The last thing we do in the loop is wait half a second just to make things less busy. So far though, we've drawn nothing on the LEDs. That's all done in drawvals() :

void drawVals() {  
    matrix.fillScreen(0);

    int high=0;

    for(int i=0;i<5;i++) {
high=max(high,max(insertops[i],max(updateops[i],deleteops[i])));  
    }

    high=int(((high+5)*10)/10);

The first thing the drawing function does is clear the virtual screen we are drawing on. Then we run through the recent values we have received over the serial port and find the highest, then round it up to the nearest 10. We'll use this value to scale the bars. Next we draw our three bars, using the most recent value for insert,update or delete and our scale factor. The three other values denote red, green and blue.

 drawCol(4,insertops[0],high,0,1,0);
 drawCol(3,updateops[0],high,0,0,1);
 drawCol(2,deleteops[0],high,1,0,0);

Then in one of the other two columns, we draw an indicator of how scaled the bars are. The more lights lit, the larger the value in the bars. This lets us see at a glance if something out of the ordinary is happening too:

    matrix.drawPixel(0,0,matrix.Color(255,255,255));
    for(int j=0;j<map(high,0,100,0,8);j++) {
matrix.drawPixel(0,j,matrix.Color(255,255,255));  
    }

Finally, we move all the pixel illuminations to the live display:

matrix.show();  
}

That only leaves the drawCol() function which draws brightly lit LEDs according to the scale and, where there's not quite a full bars worth, a dimmer LED to represent that:

void drawCol(int column, int val, int scale, int basered, int basegreen, int baseblue)  
{
  double scaledled=scale/matrix.height();
  double scaledval=val/scaledled;
  int t;

  for(t=0; t<int(scaledval); t++)
  {
    matrix.drawPixel(column,t,matrix.Color(basered*255,basegreen*255,baseblue*255));
  }

  if(scaledval!=float(int(scaledval))) {
    matrix.drawPixel(column,t,matrix.Color(basered*64,basegreen*64,baseblue*64));
  }
}

And thats all the code for the Arduino. Compile and upload that and then start the Node.js code running and you should get a set of illumination something like this depending on how active your MongoDB database is of course. Sound effects not included:

Beyond the bars

This is a simple visualisation – partly to make it easy to understand and partly to keep the cost down – but there's so much more to explore. With appropriate power supplies to the LEDs, you could chain numerous strips together to give you an ambient status display in your office. You could attach a screen shield to the Arduino instead to give an LCD panel with graphs and detailed status content.

You could break the USB chain between the Arduino and the Node.js system with a low-cost wireless connection allowing you to site the device anywhere you can get power to it - or make it battery powered (though batteries do get consumed quickly with lit LEDs). With wireless though you could strip down to a simple OLED display and beeper and create a MongoDB aware office pager. And you could even lose the PC or Mac from the equation and gather your MongoDB data on a Raspberry Pi or BeagleBone Black which will happily run Node.js.

All you need is some imagination and a MongoDB configuration which gives you oplog access... like MongoHQ's Elastic Deployments.