NewBits for the week ending 17 March: An update to RabbitMQ, TimescaleDB's PostgreSQL based time-series database, Seagate's helium-filled 12TB enterprise drive, PostgreSQL Conf Europe announce, scaling Elasticsearch, infrastructure and tests, containers at the CNCF, Google's new JPEG compression, Native Scala and using science to find the next top language.
We gather all the database, development, cloud and data news from the past week, for you to enjoy. This is NewsBits and here are the bits.
An update to RabbitMQ's distributed messaging platform has just been released. The new RabbitMQ 3.6.7 has the usual bug fixes and updates but also has a new management plugin which works with all the nodes of the message queue, not just the one. Further details are on Github.
Development also continues on RabbitMQ 3.7 with Milestone14 recently released with many, many changes, including some breaking changes. It'll also bring the ability to extend the rabbitmqctl command using Elixir/Erlang which should make for an interesting new development space for RabbitMQ. That's expected in the remainder of the first half of this year.
Taking PostgreSQL as its foundation and building a time-series workload capable system around it, the new TimescaleDB offers the uncommon combination of SQL and a time-series database in one product. The white paper (pdf) covers how the developers scaled the database for high ingest rates and how the concept of a hypertable plays into enabling this. A single node open source version is available now under the Apache 2.0 license, with a clustered version in private beta.
Seagate's 12TB Hard Drive
Seagate have launched their latest helium-filled enterprise-grade hard disk, a 12TB drive which manages 1.5TB per platter surface, with 8 platters and 16 heads packed into a 3.5" drive. Pricing is not currently available, but it's 10TB predecessor is around $500.
PostgreSQL Conf Europe
Will you be in Warsaw in October? You might be if you are into PostgreSQL. This year's pgconf.eu is running from October 24-27 in Warsaw with three days of PostgreSQL presentations. Full details to come.
Real life tales of scale
This week, we saw the Discord developers explain how they index billions of messages using Elasticsearch, the Segment team explain the economics of scale as it affected them growing an infrastructure business, and the Databricks developers who tracked down a bogus trillion rows per second problem when testing their code.
Rkt, containerd and the CNCF
The Cloud Native Computing Foundation looks to be building up an arsenal of useful cloud technologies with Kubernetes, Prometheus, OpenTracing, Linkerdm gRPC, CoreDNS and Fluentd currently under their wing. Now, there are recently proposed donations for rkt and containerd. Rkt is CoreOS's container engine and containerd is Docker's container runtime. What the CNCF decide to do with the two very comparable technologies should be known in the coming weeks.
Google's JPEG compressor
Squeezing every unnecessary bit out of your data is always useful in a resource constrained environment like the web or a database. The Google research team just announced Guetzli, a JPEG compression engine which manages to crunch JPEG files up to 35% smaller without breaking ruining the image. The technique is based on previous work done with PNG file compression. The compressor is described as "very Google" by one Twitter user.
Scala has been bound to the JVM since its inception but that may be starting to change with the announcement of scala-native 0.1 which uses LLVM's compiler infrastructure to produce native, ahead of time optimized, executables. The 0.1 release claims support for the whole Scala language, native code interop and integration with existing Scala tools. Find out more at scala-native.org.
What language is going to be big next? It's the kind of problem that needs some science. Erik Bernhardsson's approach to this problem looked at posts about why people moved from language X to language Y, map it as a grid, turn the grid into a stochastic matrix and find the eigenvectors... It's a good discussion starter and full of interesting ideas.
If you have any feedback about this or any other Compose article, drop the Compose Articles team a line at email@example.com. We're happy to hear from you.