NewsBits: Ottertune tunes databases with machine learning


NewsBits is Compose's roundup of the past week's database and developer news:

This is NewsBits: news selected by Compose's technical content curator from the world of open source databases and development, delivered as delicious chunks of information.

Ottertune's ML driven database tuning

An interesting article on Amazon and an intriguing paper on Ottertune(pdf), shed light on a research project at CMU which is applying machine learning to database tuning. Using Tensorflow and Python code, Ottertune builds ML models based on how the database reacts to various "Knobs" being adjusted and uses that to work out what tuning would be appropriate to the current scenario.

Ottertune's ML looks for important knobs to adjust, filters out irrelevant metrics and keeps learning over time. The paper also talks about other work in machine learning and optimization of databases so it's a good place to start reading about what might be tuning your database tomorrow. According to the paper the researchers have shown "that OtterTune generates configurations in under 60 min that are comparable to ones created by human experts".

Citus 6.2 is more PostgreSQLy

Citus 6.2 is the latest version of the multi-tenant distributed database built around PostgreSQL to be announced. It's aimed companies doing B2B customer analytics so new features like SQL support for interactive segmentation and funneling. The developers have also brought the databases SERIAL column handling in line with how PostgreSQL does it and Rails expects it, hiding the sharding distribution for a whole class of apps; other changes follow a similar pattern of making Citus systems behave more like standalone PostgreSQL. Implemented as a PostgreSQL extension, Citus does seem to be a demonstration of how much you can do with such extensions.

Zson 1.1

Talking about PostgreSQL extensions, ZSON 1.1 has just been released. ZSON is a JSONB compressor which can learn from data in a JSONB field's contents and create a compression dictionary appropriate to whats stored in the documents.

PostgreSQL's Explain for Rubyist

And before we leave the subject of PostgreSQL, Ruby developers may appreciate Rubyists Guide to PostgreSQL's Explain and it's Ruby centric look at how to interpret PostgreSQL's very informative EXPLAIN option.

Other updates

RabbitMQ 3.6.10 landed as a maintenance release with a good set of fixes. Be aware though, that there is a pre-release milestone of 3.6.11 in the works, and there's still that Q2/Q3 release of RabbitMQ 3.7 on the horizon -

Rust 1.18 - the book

Rust 1.18 is the latest, regular release of the Rust language and it brings a variety of changes in the fast moving language such as a restrictable pub keyword to control public visibility of variables. What's more interesting, if you haven't delved into Rust yet, the second edition of "The Rust Programming Language" is nearly complete and has been developed and written in the open. There's one chapter to go - that'll arrive with Rust 1.19 whenn the book will also get a paper edition - but unless you are a particularly quick reader, there's plenty to dig into.

Node 8.1.0 - the fast update

Fast and regular releases - something that Node does too, and here's Node 8.1.0 which brings the first swathe of tweaks and fixes to last weeks Node 8.0 release. There's also an update to the Long Term Supported Node 6 in the form of Node 6.11.0.

PyPy gets closer to boosting Python 3.5

PyPy, the JIT compiling Python interpreter, is now at version 5.8 which continues work on its Mozilla sponsored PyPy for Python 3.5. That work is now rated as beta quality and both it and PyPy for Python 2.7 are being pulled from the same codebase. Right now though, the PyPy for 3.5 only runs on 64-bit Linux.

Java delay goes ahead

Java JDK 9 is now officially pushed back, to allow the various parties involved to come up with a solution to issues around the Jigsaw module system. The new GA date is September 21st. There are some neat features coming in Java 9 - Jshell brings a REPL to the JVM, there's full HTTP 2.0 client support, new default garbage collector, and interprocess communications without dropping to native code. Oh, and the Applet API should finally be gone.

If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at We're happy to hear from you.

Dj Walker-Morgan
Dj Walker-Morgan was Compose's resident Content Curator, and has been both a developer and writer since Apples came in II flavors and Commodores had Pets. Love this article? Head over to Dj Walker-Morgan’s author page to keep reading.

Conquer the Data Layer

Spend your time developing apps, not managing databases.