Splunk Enterprise 6.6 New Features – Part III
We continue to explore the new features of Splunk 6.6. In part I and part II we talked about the new Knowledge Object management feature and the Search Editor enhancements. In this post, we will discuss the introduction of the new Search Head Cluster (SHC) graphical user interface and Indexer Clustering improvements in Splunk 6.6.
The Search Head Clustering enhancements finally provide some welcome administrative capabilities from within the Splunk user interface directly, while the index clustering enhancements offer more stability. We will focus on two new clustering features: the new Search Head Clustering interface and rollback of Indexer configurations.
Search Head Clustering – New User Interface
Splunk 6.6 now allows user to view the status of their SHC through the main Splunk user interface. Any user with admin capabilities is now able to navigate to Settings > Search Head Clustering.
From here, admins can review the name of the host, status, role and last heartbeat sent to the captain. The heartbeat column can be used to troubleshoot and check how often the members are communicating with the Captain, allowing admins to explicitly check for latency between SHC members. Delayed communication between members could be a result of improper configuration or certificates, low network bandwidth or busy TCP/UDP ports among many issues.
Additionally, the SHC interface provides the ability to ‘Transfer Captain’, giving the user easy control over which member becomes captain. In Splunk 6.5 you could only transfer captaincy through the command line, but having this feature now through the user interface, makes it more accessible and increases productivity. For example, you might want to transfer the captain to another member in order to perform some maintenance on the existing captain.
Indexer Clustering – Rollback Configuration
A great new feature in Splunk 6.6 is the ability to rollback an indexer to a previous configuration, in order to recover from operational errors or node failure. The rollback command can be executed from the Cluster Master (CM). This feature is helpful in situations where pushing configurations may result in inaccurate parsing and manipulation of data or where the configuration causes unexpected errors on the node that could have been overlooked during validation.
To test this feature, I added a new app to the master-apps folder on my Cluster Master to deploy to the indexer cluster. From the $SPLUNK_HOME/bin directory we execute:
splunk validate cluster-bundle
Validation is useful to ensure bundle configuration to apply across all peer nodes without any problems. Once executed we can check the status of the bundle below to show the results of the validation. This command provides visibility into problems that could encountered during validation. In certain instances, it details out the problem.
splunk show cluster-bundle-status
Next we will apply the bundle out to the indexer cluster using the following command.
splunk apply cluster-bundle --answer-yes
After the new bundle is applied to the index cluster, we can verify if it succeeded by using the same bundle status command again:
splunk show cluster-bundle-status
(For information on each of these commands visit http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Updatepeerconfigurations
Comparing the active bundle & latest bundle in the indexers with the one on the Master Node, the checksum of the active_bundle should be equivalent. Alternatively, you can also navigate to your indexers directory – $SPLUNK_HOME/etc/slave-apps to verify if the new app has been pushed. Now lets rollback the new configuration that we just pushed. Using the following command, the previous configuration (prior to adding the new app) is pushed to the indexers:
splunk rollback cluster-bundle
We can verify changes and status of the bundle by running the bundle status command again:
splunk show cluster-bundle-status
The active_bundle heading will indicate the checksum and the timestamp of the previous bundle in this case.
Looking to expedite your success with Splunk? Click here to view our Splunk service offerings.
© Discovered Intelligence Inc., 2017. Unauthorised use and/or duplication of this material without express and written permission from this site’s owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Discovered Intelligence, with appropriate and specific direction (i.e. a linked URL) to this original content.
