Compose's Little Bits 42 - PostgreSQL, Spark, Github's gh-ost, Thinker, Terraform, Vizcreal, Protobufs, Atom and more


PostgreSQL's past and future plan, Apache's second Spark, Github's gh-ost and database migration, cloning RethinkDB databases with Thinker, Terraform's importing, Vizcreal's visualization, Protocol Buffers updated, Atom's new display layers, Dashing Dash and Seymour Papert, Logo's creator, remembered - Compose's Little Bits are the links that have come to the attention of its Technical Content Curator, brought to you every Friday.


PostgreSQL and Uber - In the last Little Bits, we mentioned Uber Engineering's migration from PostgreSQL to MySQL. The PostgreSQL community, demonstrating what's so good about it, read and made considered responses in a email thread which looked at what had already changed about PostgreSQL and what still was to be addressed.

PostgreSQL 10 Planning - While its now established that PostgreSQL 9.6 will be version 9.6, not 10, the planning for PostgreSQL 10 has begun in earnest with the various development groups sharing their roadmaps and what is already being worked on. More parallelism, better replication, new partitioning, even smarter foreign data wrappers and of course numerous performance enhancements are on the conversation agenda.

Spark 2.0.0 - Version 2.0 of the cluster computing platform Apache Spark rolled out at the end of last month. Better SQL2003 support, CSV data sources, two to ten times performance improvements in places, experimental structured streaming and some older APIs put into maintenance mode. You can find out more about the release on Databricks blog.

Github's gh-ost - Over at Github, the engineering team have talked about gh-ost, a tool for migrating schemas online. They avoid issues around trigger based migration and parse the DDL from the server, decoupling the entire process and making it eminently testable and auditable. Best of all, they've open sourced the tool (on a github repo of course) under an MIT licence.

Thinker - Looking for a tool to clone your RethinkDB databases? The folks at Internalfx may have the tool for you in Thinker which clones and syncs RethinkDB databses. If you think you know the name InternalFX, they are the people who did Regrid, the file storage system for RethinkDB too.


Terraform - Hashicorp's Terraform has just had it's first major feature release in a year and it's a big one for the infrastructure automation platform. Additions in Terraform 0.7 include the ability to import existing infrastructure resources, use read-only data sources effectively, track resource states and move resources between states and use list and map structures to manage collections of data.

Vizceral - The Netflix developers have renamed their "traffic intuition tool" from Flux to Vizceral and open sourced it. Vizceral turns network data into a holistic, organic flow between nodes and is the flagship project of something Netflix call Intuition Engineering. Note that this doesn't gather the traffic data for you – at Netflix, it's extracted from their Atlas and Selp systems – so you'll need to take care of that first. There's more in this talk by Justin Reynolds from Monitorama.


Protocol Buffers 3.0.0 - Google's Protocol Buffers, aka Protobufs, have established themselves as a reliable way of serialising data between applications. Now, Protocol Buffers 3.0.0 has arrived with more languages officially supported (C#, JavaScript, Ruby, Objective-C), lots of cleanups and even a canonical representation in JSON for when you need to get to places Protobufs can't reach. Don't panic though, Protobufs 2 is still supported.

Atom 1.9 - The Atom editor from Github moves on with the release of version 1.9 which comes with new "display layers" which will allow smarter content handling inside the editor and drag and drop layout which makes multiple pane handling a doddle.


Dash Dash - If you consider making an existing bit of consumer hardware to bend to your will, consider this cautionary tale of how one man found he was burning time trying to make Amazon Dash buttons do his bidding and how he found redemption through the soldering iron.


Seymour Papert - The Logo Foundation tweeted the news that Seymour Papert, co-creator of the Logo language which introduced many to programming for the first time, had passed away at the age of 88. A co-founder of the MIT Artificial Intelligence Laboratory with Marvin Minsky, he created the Logo language based on the work Jean Piaget as a dialect of Lisp and a tool for learning.

Dj Walker-Morgan
Dj Walker-Morgan was Compose's resident Content Curator, and has been both a developer and writer since Apples came in II flavors and Commodores had Pets. Love this article? Head over to Dj Walker-Morgan’s author page to keep reading.

Conquer the Data Layer

Spend your time developing apps, not managing databases.