Redis Counting with INFO - Noteworthy at Compose

Published

This is your weekly summary of Compose news for those changes and updates which can make your life easier. In this edition, we take another pass at counting all your keys with Redis. And also take a browse through the past week's Compose Articles.

Redis Key Counting Redux

In last week's Noteworthy we mentioned that people had performance trouble when counting the number of keys they had in Redis with KEYS *.

We were reminded, following that article, that we should have mentioned that there is actually a quick way to count the total number of keys in Redis. The DBSIZE command will return the total number of keys in a Redis database.

> dbsize
(integer) 27
>

In the process of discussing DBSIZE, the issue of other key statistics came up. How do you find out how many keys will expire? And how soon? For these questions, we turned to...

Redis INFO

The INFO command in Redis is a great source of human readable information about how the Redis server is running. It returns multiple sections of data on the platform, clients, memory and statistics. One of the sections is the Keyspace section which includes information about keys and their lifetimes. You don't have to retrieve all the INFO to get this data. You can use INFO KEYSPACE command retrieves only the keyspace section:

> info keyspace
# Keyspace
db0:keys=27,expires=0,avg_ttl=0  
>

The thing is that the result from the INFO command is, in most Redis drivers, a big string. So you have to parse it into something useful. By way of illustration, here's a snippet of JavaScript code which breaks up the keyspace data.

server.info("keyspace", (err, result) => {  
    lines = result.split(/\r\n/);
    dbstats = {};
    lines.forEach((line) => {
        if (line.length>0 && !line.startsWith("#")) {
            parts = line.split(/[\:\=\,]/);
            value = {}
            value[parts[1]] = parseInt(parts[2]);
            value[parts[3]] = parseInt(parts[4]);
            value[parts[5]] = parseInt(parts[6]);
            dbstats[parts[0].slice(2)] = value;
        }
    });

    console.log(dbstats);
    console.log(dbstats[0].keys);
});

It first breaks the result into lines, then for each line breaks it up into the keys and values. Then it reassembles them into a single object which is stored in the dbstats variable. This produces a result like so:

{ '0': { keys: 27, expires: 0, avg_ttl: 0 } }
27  

Now the number of keys is the obvious value we get from parsing the results of INFO KEYSPACE but what are expires and avg_ttl? Well, expires is a count of the number of keys with an expiry time set on them. The avg_ttl, or average time to live, is the average time in milliseconds that those keys have to live. Let's create a key with an expiry....

> set example 10000 EX 100
OK  

And we see...

{ '0': { keys: 28, expires: 1, avg_ttl: 97149 } }
28  

More Keyspaces, More Counting

One under-appreciated aspect of Redis is that it supports multiple keyspaces, also confusingly called databases. Applications default to keyspace 0 but if you add "/1" to the end of your Redis URI (or specify -n 1 in the redis-cli command line) you can access keyspace 1. You can also switch between them with the [SELECT](https://redis.io/commands/select) command. Redis on Compose is configured to support 16 of these keyspaces. Now, it's recommended that you don't try and use this to host multiple unrelated applications on the same server, but instead use it as a way to control your applications use of keyspace. For example, have the session management in one keyspace, device identifiers in another, job queues in another and so on.

If we SELECT into some new keyspaces, set some keys and run our example, we can see this:

{ '0': { keys: 27, expires: 0, avg_ttl: 0 },
  '1': { keys: 3, expires: 0, avg_ttl: 0 },
  '15': { keys: 4, expires: 1, avg_ttl: 9991549 } }
27  

We can quickly see key counts for all the spaces and de-aggregate the Redis statistics for our application without injecting prefixes and queries into our engineering. If you use multiple keyspaces, using INFO is a quicker way to gather the statistics for all of them with just one command and a little bit of code.

Compose Articles

Last week at Compose saw news of an update for PostgreSQL which is now available on Compose. The new versions will become the preferred version tomorrow, July 31st. We also had our weekly delivery of NewsBits which covered Elasticsearch releases, 3D rendering of graph databases, interesting Redis tools and the latest Ubuntu update.

That's it for this week's Noteworthy at Compose. Onwards to next week!

Dj Walker-Morgan
Dj Walker-Morgan is Compose's resident Content Curator, and has been both a developer and writer since Apples came in II flavors and Commodores had Pets. Love this article? Head over to Dj Walker-Morgan’s author page to keep reading.

Conquer the Data Layer

Spend your time developing apps, not managing databases.