Debugging Lua in Redis

A little while ago, we published a speed guide to Redis's Lua scripting. It's super useful stuff and almost immediately, the question was asked "But how do we debug these scripts?". Until Redis 3.2, the answer was "Very, very carefully" but Redis 3.2 brought with it a Redis Lua debugger so let's have a quick look at it.

How to start debugging

Let's jump back to the speed guide where we went to run a script which incremented various keys and counted them:

$ redis-cli -h sl-eu-lon-2-portal.1.dblayer.com -p 10030 -a secret --eval broadcast.lua region:one
(integer) 33
$

That's not right. Let's invoke the debugger by adding --ldb to the command line:

redis-cli --ldb -h sl-eu-lon-2-portal.1.dblayer.com -p 10030 -a secret --eval broadcast.lua region:one  
Lua debugging session started, please use:  
quit    -- End the session.  
restart -- Restart the script in debug mode again.  
help    -- Show Lua script debugging commands.

* Stopped at 1, stop reason = step over
-> 1   local count=0
lua debugger>  

And we're into the debugger. The essential commands are s/step, n /next and c/continue. You can use the first letter of the command or the full command (nothing in between though). If we type s and step now:

lua debugger> s  
* Stopped at 2, stop reason = step over
-> 2   local broadcast=redis.call("lrange", KEYS[1], 0,-1)
lua debugger>  

We move on in the debugger, we can see why the debugger stopped again (it stepped over) and the line it is now sitting on. Stepping again will show us something very useful:

lua debugger> s  
<redis> lrange region:one 0 -1  
<reply> ["count:emea","count:usa","count:atlantic"]  
* Stopped at 3, stop reason = step over
-> 3   for _,key in ipairs(broadcast) do
lua debugger>  

The command calls on Redis to do a lrange command. The debugger shows the command sent to Redis (<redis>) and the result that came back from it (<reply>), before stopping on the next line. The l/list command will show us the code around the current line:

lua debugger> l  
   1   local count=0
   2   local broadcast=redis.call("lrange", KEYS[1], 0,-1)
-> 3   for _,key in ipairs(broadcast) do
   4     redis.call("INCR",key)
   5     count=count+11
   6   end
   7   return count
lua debugger>  

This is a tiny script, so we get to see it all. You can give a line number or a range of line numbers to l to see particular parts of the script. If you want to see the whole script you can say w /whole.

Setting breakpoints is an essential part of any debugger. The b/break command handles all of that. Let's set a breakpoint on line 5 by typing b 5.

lua debugger> b 5  
   4     redis.call("INCR",key)
  #5     count=count+11
   6   end
lua debugger>  

You can list all your breakpoints with just b:

lua debugger>b  
1 breakpoints set:  
  #5     count=count+11
lua debugger>  

And clear them all with b 0. For now, we just want to continue so we enter c:

lua debugger> c  
* Stopped at 5, stop reason = break point
->#5     count=count+11
lua debugger>  

And we've stopped on the breakpoint. We can quickly check the variable count with print count and then continue on:

lua debugger> print count  
<value> 0  
lua debugger> c  
* Stopped at 5, stop reason = break point
->#5     count=count+11
lua debugger> print count  
<value> 11  
lua debugger>  

Oh no! Someone finger fumbled the +1 and added 11. Ok, this is a somewhat contrived example which relies on you not looking at the code at all. But we've located the problem and thats what counts.

There's a couple of other things that you can call on when debugging. If you don't give a variable name to the print command, it dumps all the local variables. If you've got lots of function calling taking place then t/trace will give you a backtrace. You can also evaluate some lua code using e/eval - it's run in a different environment so it won't interact with the code you are debugging. And, you can make a call on redis with the r/redis command like so:

lua debugger> redis lrange region:one 0 -1  
<redis> lrange region:one 0 -1  
<reply> ["count:emea","count:usa","count:atlantic"]  
lua debugger>  

Now, you may wonder if these changes persist and that's an interesting question. If we look at the command which controls scripting SCRIPT DEBUG we find it has three settings: NO which turns debug off, YES and SYNC.

When set to YES, changes made by the debugger session are discarded at the end of the session when you use a/abort or q/quit commands. How does this work? Well, you aren't actually talking to the server. When the debug sessions starts, Redis forks a copy of the server for your debug session to talk to so you won't block the running Redis server.

The SYNC option, as you may guess, doesn't do that and all the changes are made to the server. It WILL bring your server to a grinding unavailability if you use this mode though as it blocks. It's not recommended for use unless you have a very, very, very good reason.

Finally, there are two calls you can use in your Lua code: redis.debug() lets your Lua code emit messages when in debug mode to assist your debugging efforts and redis.breakpoint() can be inserted where you want to "fix" a breakpoint in the code.

This has been a brief introduction to the Redis Lua Debugger. You can find out more in the Redis documentation and don't forget to get h/help at the debugger command line.