MQTT and STOMP for Compose RabbitMQ

Published

RabbitMQ talks AMQP 0.9.1 by design; it's a solid, well-tested messaging protocol. It's also a full featured protocol which makes it overkill for some scenarios and that's why RabbitMQ also has support for incoming MQTT and STOMP messages. Today, we're pleased to announce you can make use of that support on Compose's managed RabbitMQ with two new Add-ons.

What's MQTT?

MQTT is a lightweight, binary, messaging protocol which started life as a way for remote IoT sensors to reliably send messages upstream. It essentially allows a device to fire off an arbitrary payload of binary data to a broker, at a named topic (like a RabbitMQ routing key) and let other services pick it up and parse it. It has turned out to be really useful in many other realms. So much so it became an OASIS standard and there are lots of supporting libraries out there.

What's STOMP?

Simple Text Oriented Messaging Protocol, (STOMP), is a text based protocol modeled on HTTP for interchanging data between services. Modeled on HTTP means that you can practically telnet into a STOMP server to send headers and text payloads. It's a simple but powerful option and also has many implementations.

Why send to RabbitMQ?

Both MQTT and STOMP have brokers or work with other message brokers. If you use RabbitMQ as your messaging platform for your applications, it still makes sense to bring these other protocols under the wing of its persistent, managed and reliable handling. With the MQTT Add-on, messages from IoT services can be reliably ingested and forwarded on to services that can use them. The STOMP Add-on does the same with infrastructure tools and other applications, bringing them into your application's domain with the least fuss.

Full details of how the adapters work on the RabbitMQ side are in the RabbitMQ documentation for MQTT and STOMP. We've configured the portals and adapters to allow the maximum flexibility to users and developers; there's no hardwired behavior beyond the standard defaults. Both, though, do exclusively use SSL/TLS connections for data privacy in transit. Each portal is $4.50 a month.

Using RabbitMQ and MQTT

As an example, we'll look at generating some MQTT packets and then get them into a Compose RabbitMQ...

Starting the MQTT Add-on

Head over to the Compose console for your RabbitMQ deployment and you will find two new entries in Add-ons; the MQTT and STOMP Add-ons. These are custom configured portals; click Add on the MQTT Add-on and you'll be asked to confirm that it should be created. Click Create to confirm and... well, that's it. We like to make things easy. You'll most likely want the URL to connect to, so head back to the Add-ons view and select the Configure button that is now visible on the MQTT Add-on.

An MQTT Configuration

Copy that URL. And if you haven't yet, get a user set up on your RabbitMQ Vhost. Now we're ready to get started sending some MQTT messages...

Sending some MQTT

The simplest way to generate an MQTT packet for this is it uses the MQTT.js package for Node. Not only is it a fully-fledged library but it also has a command mode. Assuming you have Node.js installed run npm install mqtt -g to get the mqtt command added. Now let us use that to send a simple payload:

mqtt publish -C mqtts -u user -P password -h mqtt186-2.bigwig.compose-3.composedb.com -p 15477 -t "compose"  -m "From MQTT.js"  

Most of this command's arguments give the details on how to connect and are taken from the Add-on configuration. The -C takes the protocol, in this case mqtts as we are doing MQTT over SSL. Then it's -u and -P for the RabbitMQ username and password. Next is -h for the hostname to connect to, followed by the port -p on that host. And, that's the connection credentials done. Now we move onto the MQTT side of things.

The -t sets the topic for the message and the -m sets the payload for the message. Running the example command would send the message "From MQTT.js" to the topic "compose". From the MQTT side that's pretty much it.

MQTT in a RabbitMQ world

Things on the RabbitMQ side are different. With the MQTT Add-on running, you'll notice an exchange called amq.topic.

The amq.topic

Every incoming MQTT message passes through this exchange. Now, the "topic" of the MQTT message becomes the routing key of the AMQP message. That means we can create a queue and bind it to amq.topic to pick up the MQTT topics we want. So we'll add a queue:

Create a queue

And then, back at the amq.topic exchange we'll bind that queue to the exchange:

Binding Compose

Now, we can run the mqtt publish... command from earlier and if we're watching the compose queue we'll see something like this:

The MQTT message arrives

Let's just make sure and check whats in the queue:

Checking the queue

And there's our string payload, "From MQTT.js" at the end.

This has just been the simplest example possible from the command line of publishing into RabbitMQ. It's also possible to subscribe to queues/topics too. This command will subscribe and listen to our compose queue:

mqtt subscribe -C mqtts -u user -P password -h mqtt186-2.bigwig.compose-3.composedb.com -p 15477 -t "compose" -v  

When a message is placed in the compose queue, it'll get picked up by this subscriber and print out the topic and payload on the console. To take this one step further, we'll now wrap both those operations together in one simple Node application:

var mqtt = require('mqtt');  
var client  = mqtt.connect('mqtts://user:password@mqtt186-2.bigwig.compose-3.composedb.com:15477');

client.on('connect', () => {  
  client.subscribe('compose')
  client.publish('compose', 'This is from an MQTT.js using Node application')
})

client.on('message', (topic, message)  => {  
  console.log(message.toString())
  client.end()
})

client.on('error', (error)  => {  
    console.log(error);
})

This code starts an MQTT connection. When the connect event fires it first subscribes to the compose topic/queue for updates and then publishes a message to the queue. As it's subscribed to the topic, when that message is published to subscribers, it generates a message event. Here, that even prints the message payload and then shuts down the client, letting the app exit. If anything breaks, the error event is fired.

It's that simple.

A STOMP example

You'll find the credentials for a STOMP connection in the STOMP Add-on configuration screen. Again, there are a lot of libraries you could use to connect to it; all they need to be specifically able to do is STOMP over SSL. For an example, we'll use the Stompit Node package. Here's some code which does almost the same thing as our MQTT example:

var stompit = require('stompit');

var connectionOptions = {  
    'host': 'stomp187-2.bigwig.compose-3.composedb.com',
    'port': 15619,
    'connectHeaders': {
        'host' : 'bigwig',
        'login' : 'user',
        'passcode' : 'password'
    },
    ssl: true
};

This is the essential part; the connection options. You'll need to break up the URL into host and port and have your RabbitMQ username and password to hand. Most of the values are obvious; the ssl:true activates the connection's encryption. The odd part is that host in connectHeaders; that's a STOMP host which should be set to the vhost name on the RabbitMQ server. We can now connect and send a STOMP message:

stompit.connect(connectionOptions, (error, client) => {  
    if (error) {
        console.log('connect error ' + error.message);
        return;
    }

    var sendHeaders = {
        'destination': '/queue/compose',
        'content-type': 'text/plain'
    };

    var frame = client.send(sendHeaders);
    frame.write('From Stompit');
    frame.end();

The STOMP adapter in RabbitMQ lets us send directly to a queue or an exchange. Here, we send straight to our compose queue. We also set a content type for the STOMP message and literally write the payload in before finish the STOMP message, or frame as STOMP refers to it. Now we can go and subscribe:

    var subscribeHeaders = {
        'destination': '/queue/compose',
        'ack': 'client-individual'
    };

The ack setting in the hash lets us manually acknowledge messages. We use that to subscribe:

    client.subscribe(subscribeHeaders, (error, 
message) => {  
        if (error) {
            console.log('subscribe error ' + error.message);
            return;
        }

        message.readString('utf-8', function (error, body) {
            if (error) {
                console.log('read message error ' + error.message);
                return;
            }
            console.log('received message: ' + body);
            client.ack(message);
            client.disconnect();
        });
    });
});

Once subscribed, a received message will trigger the callback, where the content of the message payload will be printed, the message acknowledged (removing it from the queue) and then the client is disconnected and the app exits.

Add MQTT or STOMP today

If you've been looking for a way to bring disparate messaging together, then these two new Add-ons could be right for you.


If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at articles@compose.com. We're happy to hear from you.

attribution Peter Lloyd

Dj Walker-Morgan
Dj Walker-Morgan is Compose's resident Content Curator, and has been both a developer and writer since Apples came in II flavors and Commodores had Pets. Love this article? Head over to Dj Walker-Morgan’s author page to keep reading.

Conquer the Data Layer

Spend your time developing apps, not managing databases.