Kibana and Compose Elasticsearch

We're going to be talking about how you can run your own Kibana instance for Elasticsearch. You may think this is an odd thing to want to do, but all Elasticsearch users should be paying attention to this.

That's because Elasticsearch has been restructured over the last few versions, so some components have been moved away from the central server's plugin architecture and given their own server. That process isn't slowing down either. The future doesn't look good for the other site plugins like Head and Kopf. While the decisions are architecturally sound, it does mean that we at Compose are tracking an evolving application. We're working away on how to incorporate the changes in Elasticsearch in a smooth an effective way and we'll be letting you know when we have more to say on that.

In the interim though, you may just want to run Kibana so you can explore your database with dashboards. You can install Kibana locally or up on a cloud instance; we'll look at setting it up locally though the steps should be the same for a cloud instance.

Get your Kibana on

To start this process we'll need the appropriate binary for Kibana. This isn't as easy as going to https://www.elastic.co/products/kibana and clicking Download though because Kibana versions are version matched with Elasticsearch versions. Clicking Download there would give you, at time of writing, Kibana 4.5.0 which is compatible with Elasticsearch 2.3.x and nothing earlier. So, where to look? Well, the same page also lists Kibana 4.1.6 which is compatible with Elasticsearch 1.4.4 to 1.7 and if you are running one of those versions, you can use those.

But for anything between 1.7 and 2.3.x, you'll need to dive into the Elastic "Past Releases" link on the page, and here. For Elasticsearch 2.2, you'll want a Kibana 4.4.x, currently version 4.4.1. For Elasticsearch 2.1.x, which is what's currently available on Compose, you'll want Kibana 4.3.x, currently version 4.3.3.

So let's download Kibana 4.3.3 for our Compose Elasticsearch 2.1.1 installation. You'll find the available downloads on the release notes pages. We're going to install on Mac OS X and that means we end up downloading kibana-4.3.3-darwin-x64.tar.gz . Unpack this file, either with a double click in the desktop or a tar xvZf - kibana-4.3.3-darwin-x64.tar.gz at the command line. You'll now have a `kibana-4.3.3-darwin-x64 directory. Go into that directory and have a look:

tar xvZf - kibana-4.3.3-darwin-x64.tar.gz If this looks familiar, it may be because Kibana is now a node.js application. If you started it up right now though – by running bin/kibana – it would try and connect to a local instance of Elasticsearch. We want it to connect to a Compose deployment though. So, first step is to get our connection strings from Compose Elasticsearch.

Get your Compose on

Go to the Compose web console at app.compose.io and log in. If you don't have an account why not sign up for the 30 day free trial. Once logged in, select the Elasticsearch deployment and you'll see this:

You'll want to have added a user, ideally one specifically for your new kibana instance, in the Users panel. Then you'll need one of the HTTP connection strings from the Connection info panel. With that information in hand, return to your command line where your Kibana software is being set up. You want to edit the config/kibana.yml file so run your preferred editor. We'll use Atom:

The line we're interested in is commented out but begins # elasticsearch.url: . Find it and remove that # . Next change the value of the URL to the one from the Compose console, substituting in the username and password of your "Kibana user" into it. Make sure the URL is surrounded by double quotes. Ok. Once you've done that we're ready to start it up.

Run bin/kibana and you should see something like this whizz by:

It's the Server running at http://0.0.0.0:5601 that shows the Kibana server is up and running. Now you can connect to the server. Open up your browser and go to http://localhost:5601 and you'll be able to connect.

If you are new to Kibana, your next stop should be picking up Elastic's getting started guide at this point. If this is a completely fresh install, remember to load Elasticsearch with example data so you have something to work with.

Then you can move on the Discover, Visualize and create Dashboards for your data.

Remember, if you are deploying into a cloud environment, take steps to lock down access to Kibana to either authorized users or IP addresses depending on the tools available in your cloud instance. You'll find instructions on how to deploy onto Ubuntu, RHEL and Centos on Elastic's Getting Started page. Note what we mentioned about matching versions and adjust the current 4.5 version number in URLs to 4.3.3 to match Compose's Elasticsearch deployments.

This should get you up and running with your own Kibana and ready to dig into your data in your Compose Elasticsearch deployment. Have fun in there!