Compose's Data Browser comes to Elasticsearch

Today, we're excited to introduce you to the new Compose Elasticsearch data browser that makes your Elasticsearch deployment even easier to manage from the web. You now have immediate access to the Elasticsearch API without leaving the Compose Console.

This means you will not have to depend on curl and other tools to keep an eye on indices and execute queries. Using the Compose data browser from the web means there's no need to remember how to set up a connection and memorize all your indices, types and URIs. In the data browser, you can browse the indexes, dig through the types and get down to the data which you can edit and update. It's designed to complete your digital toolbox with a quick to access, simple and powerful interface.

This is our first data browser for Elasticsearch and we'd like to take you on a tour of what capabilities it already offers.

The Data Browser

Let's start our tour by logging into the Compose browser and selecting Browser from the sidebar menu. Here, we'll see the index shakespeare created from the Elasticsearch Shakespeare dataset.

In this view we'll see the deployment's "status", "uptime", total disk "size", number of "nodes" and "shards", and the total number of "documents" from all indices. Below that, the shakespeare index displays its disk "size", the "count" of the documents in the index, and the number of documents that have been "deleted". Let's click on the shakespeare index to see what's inside:

Here, we'll find the types belonging to the index. From here, we'll be able to select a type to edit and query documents corresponding to that type, or we could create more types by clicking the Rest editor link. The sidebar REST and Admin buttons take us to our editor and administration panel. However, we'll start from the top-down by looking at the data within a type and moving on from there, so let's start by clicking the line type where we'll get the following:

This is the Documents view for the line type. We'll see the Elasticsearch URI search request in the console that gives us by default the first 10 requested documents and their contents, which are displayed in the Query results table. We can edit the search request in the console by selecting different parameters for q, editing the default size of 10 documents per page, adjusting where the query should offset from , and filtering the _source. Filling out the search request, we get something like the following:

Clicking on any of the returned documents will take us to a document editor where we can modify the content like this shown here:

The document id, the number next to line at the top, is not included in the document's content because it's being used to select it. This editor lets us Validate, Save, and Delete content. We can also Revert any changes made as long as they haven't been saved. Validating the JSON can be done by clicking Validate, or our data will be validated when saving at runtime because invalid JSON won't be stored and will flag an error.

Going back to our Documents view, we can also insert documents by clicking Insert Document in the right-top corner.

This will take us to an editor that shows where we can insert a new document. It shows the title New Document at the top that's displayed here:

You can enter any valid JSON data within the editor and it will be validated either by clicking Validate or automatically upon saving. If you do not supply a document _id, one will be created for you like this:

Going back to the Document view, we'll see the Mapping tab. Clicking the tab takes us to the Mapping view where the mapping types and field datatypes are displayed:

Here, we'll see the mapping types and field datatypes related to line. The Mapping view only shows us the fields and datatypes belonging to the type.

Going back to the sidebar menu, let's go to the REST Editor by selecting the REST button taking us here:

The editor allows us to access the Elasticsearch API selecting one of the GET, PUT, POST, and DELETE methods and by inserting a URI endpoint into the input field shown here:

The editor is not a substitute for the command line. However, it makes it easy to access data from anywhere on the web to execute simple queries and insert, update, or delete documents and indices using the Elasticsearch API running curl.

Let's move now to the last button on the sidebar, Admin. Selecting it will take us to the administration page displaying "tasks" buttons shown here:

All of the "tasks" are performance and optimization APIs from the Elasticsearch Indices API. Their effects only apply to the index were in, and not all the indices of the deployment. At the bottom there is the button "Delete Index", which will drop the index including its data from the deployment. Remember that before deleting an index, you should make all necessary backups of your data because you can't undo it.

Deleting all your indices will start you off with a fresh browser view that looks like the following:

To create an index we have two options: use curl or the browser. For demonstration purposes, we'll use the browser which has a Create Index button on the top-right corner. This opens a console window that shows the same curl command we'd use to create an index:

The index fields can be edited and are configured by default with 5 shards and 1 replica. Adding a name and clicking Run will create the index and take us to the Index view showing our types:

As we saw before, this view is where our types are collected. Types can be added from the browser by selecting the REST editor link in the blue bar or REST on the sidebar - both take you to the REST Editor. Selecting the REST editor link, however, is better because it will bring up the REST Editor with a generated URI endpoint and a JSON mapping template to create type and mapping properties as shown here:

The template preloads the current index path with the _mapping endpoint from Elasticsearch's PUT mapping API. To successfully Make Request we must fill in [type name here] and add mapping properties where it says "[field name]". Filling in the template with Elasticsearch's Shakespeare dataset default type and mapping would look like the following:

Before running Make Request we could run Validate JSON, but the editor will flag any errors at runtime, which means that it automatically validates JSON. Any errors or successful requests will be shown in the Results box below the editor. A successful request to create a type gets following result:

Now, going back to the Index view by clicking Types from the sidebar will show us the default type created:

If we want to upload data via the browser editor, we could by selecting REST on the sidebar. This will take us to the REST editor where we can use Elasticsearch’s Bulk API like this:

While it's possible to upload datasets via the editor, it's not recommended. Uploading large datasets in the browser could either slow it down, or crash the browser altogether. So, while the editor has the ability to _bulk insert our data, a more optimal way is just to use curl to connect to our Compose deployment and upload data. We can do that by executing the following command in the terminal:

curl -XPOST 'https://hamlet:mypass@aws-us-east-1-portal383.dblayer.com:38847/shakespeare/_bulk' --data-binary @shakespeare.json  

From here, go back to the main browser page where we'll see the shakespeare index populated with the uploaded JSON documents.

Finishing Up

You can see there are a number of features in the data browser that provide access to many of Elasticsearch's powerful APIs, and we'll be enhancing the data browser to provide you more features in the future. With the browser, you can manage and have access to your data from the web, at any location, and at any time, and we'd like to make it your preferred tool to manage Compose Elasticsearch.

Image by João Silas