Note: This essay originally appeared here with demonstrative images and more.

Kibana took a massive leap forward last month with the launch of its fourth edition, featuring new capabilities that dramatically enhance its efficiency as well as showcase a new and improved look and feel.

Based on our work with Kibana 4 throughout its beta over the last six months, I would say Kibana 4 is a major improvement over Kibana 3. If you are wondering how this upgrade can affect your work, you’ll be happy to learn that the new interface offers some advanced tools that I am confident you will utilize and benefit from. I would like to share what we believe to be the five greatest features of Kibana 4.

Moving from Kibana 3 to Kibana 4

While Kibana 3 was useful for visual displays of Elasticsearch results, it had quite a few pitfalls including its limited flexibility in allowing users to customize dashboards and its inability to leverage some of the advanced features of the Elasticsearch engine.

Working with the beta mode of Kibana 4, we tested and explored the new Kibana features that were designed to resolve those issues and provide further flexibility. Following our experience, here are the five greatest features.

1. Search Highlights

Scanning through logs for a specific event can be tedious, especially when going through hundreds or even thousands of logs. While Kibana 3 did a great job filtering Elasticsearch results and delivering relevant data, the built-in feature of search highlights in Kibana 4 is a major visibility improvement. It allows users to arrive at relevant results and locate them instantly in a clear and convenient visual manner while scanning through relevant Logstash events.

2.  Enhanced Elasticsearch Aggregation

Unlike Kibana 3, the aggregation and calculation process in the new Kibana edition are done on the server side of the Elasticsearch cluster. This architectural change makes Kibana 4’s aggregation system far more flexible and efficient than its previous counterpart.

In addition to its enhanced calculation abilities and enhanced performance, Kibana 4 users are able to access Elasticsearch APIs in a visual way and perform a wider range of searches. New search capabilities include aggregation queries such as `distinct` to find the “top 5 repeating events? as well as sums and averages.

3. Scripted Fields

Kibana 4 allows users to create scripts and plant them within the Elasticsearch server. Once the scripts are planted, their returned results can be viewed in real time as if they were data arriving directly from the logs.

In effect, Kibana 4’s scripted fields present users with the ability to run calculations inside the Elasticsearch server. For example, if a user is interested in creating a graphic display that exhibits events based on the time of day they occurred, the user can write a relevant script using Kibana 4’s interface, then plant it into the Elasticsearch server. The script that is planted will run continuously, allowing the user to view the results on the dashboard.

4. Dynamic and Flexible Dashboard

Up until the latest edition, Kibana’s dashboard display was a fixed structure: once you defined it, it remained static. The new dashboard is flexible and dynamic; allowing users to save elements and drag-and-drop them into new dashboards. This can of course save users many hours of tedious work, defining and re-creating elements that are already in existence. The same goes for visualization. Kibana 4 allows users to save visualizations and searches without having to recreate or redefine them. (We have brought the black theme back to Kibana 4, and speaking to Rashid at Elastic{ON}, it seems like the black theme will be making a comeback to all Kibana 4 users.)

5. Separation of Client and Server

As I mentioned above, Kibana 4 is server and client based, which makes security features easier to integrate. While this change might not affect ordinary users as much, it will make a huge difference to advanced users and administrators working with the system. They are now able to provide authentication and encryption on the server layer, allowing for more robust data protection and security.

Are There Any Pitfalls to Kibana 4?

As you can gather from the list of benefits mentioned above, I believe the new Kibana edition stores an abundance of positive opportunities for users. But that being said, there are two critical issues that you should be aware of before upgrading to Kibana 4. The first is that in order to use Kibana 4, the Elasticsearch cluster must be upgraded as well (to version 1.4.3). The second is that the new edition is not backwards-compatible with earlier editions. This means that dashboards and displays built in previous editions will not be moveable into the new Kibana 4 dashboards, and you will need to manually recreate them in the new interface.

Over the last months, we’ve supported our customers with the upgrade to Kibana 4 while running Kibana 3 in parallel. This means that instead of having to manually set up the entire interface environment from scratch, we enabled them to integrate it to the new interface easily. If you are interested in the Kibana 4 upgrade, we will be happy to assist and elaborate more on how that can be done—feel free to contact us!