Smart AnSwerS #65

Hey there community and welcome to the 65th installment of Smart AnSwerS.

We have a couple back-to-back community events happening right after the upcoming long Memorial Day weekend! The next SplunkTrust Virtual .conf Session is scheduled for Tuesday, May 31st at 12:00PM PDT. SplunkTrust member rich7177 will be teaching nOObs the basics of navigating Splunk Web and, time permitting, how to build reports, visualizations, and dashboards. For those of you in the San Francisco Bay Area next week, the SFBA User Group will be on Wednesday, June 1st @ 6:00PM PDT at Splunk HQ in our brand new building next door! Come join us in the shiny new space as Sr. Engineering Manager mszebenyi, original …

» Continue reading

Configuring Nginx With Splunk, REST API & SDK Compatibility

Last year I posted an article on how to configure HAProxy with Splunk, REST API & SDK compatibility. Yesterday, I posted an article on how to configure Nginx as a load balancer in front of a tier of HTTP Event Collectors. Today, I want to iterate on the work I did yesterday and show a basic config for Nginx that’s compatible with Splunk, the REST API and SDK’s.

You’re going to need to build or install a version of Nginx that enables HTTPS support for an HTTP server.

./configure --with-http_ssl_module

If you install from source and don’t change the prefix then you’ll have everything installed in /usr/local/nginx. The rest of the article will assume this is the …

» Continue reading

Configuring Nginx Load Balancer For The HTTP Event Collector

The HTTP Event Collector (HEC) is the perfect way to send data to Splunk, at scale, without a forwarder. If you’re a developer looking to push logs into Splunk over HTTP or you have an IOT use case then the HEC is for you. We cover multiple deployment scenarios in our docs. I want to focus on a single piece of the following distributed deployment for high availability, throughput and scale; the load balancer.

You can use any load balancer in front of the HEC but this article focuses on using Nginx to distribute the load. I’m also going to focus on using HTTPS as I’m assuming you care about security of your data in-flight.

You’re going to need to …

» Continue reading

Vote using Splunk

Someone recently challenged me to use Splunk for voting. Splunk is a versatile platform, why not make a voting app? Sigi and Stephen put the app together one afternoon and then I tested it out on a live audience during SplunkLive! San Francisco.


Picture1 copy


It worked like a charm and we gained insight from the audience. That’s when I realized, although it’s not a typical use case of Splunk, this app could be useful for others. From polling an audience during a presentation or even getting consensus from coworkers on a question during a meeting, maybe I should put the app on splunkbase.


» Continue reading

Smart AnSwerS #64

Hey there community and welcome to the 64th installment of Smart AnSwerS.

One of the Splunk Cloud support engineers left on vacation last week, so in true Splunk fashion, his desk is getting a complete makeover by the time he returns! yannK has been putting on his creative hat this week to transform the desk into a Star Wars TIE Fighter which has been coming together incredibly well. If it were my desk, I’d leave it as a permanent installation because it looks that cool and is still completely functional as a work station…not that I’m jealous or anything ;P

Check out this week’s featured Splunk Answers posts:

How would one correctly configure DATETIME_CONFIG for an app that

» Continue reading

Smart AnSwerS #63

Hey there community and welcome to the 63rd installment of Smart AnSwerS.

With Splunk HQ officially more than two times larger, and Splunkers now spread out across more square footage, things have gotten eerily quiet around here as everyone is adjusting to their surroundings, getting to know new neighbors, and figuring out where all the new conference rooms are. Slowly, but surely, we’re getting comfortable in our new home, and once we’re completely settled in, we’ll find ourselves back into the groove of things with a nice balance of work and play :)

Check out this week’s featured Splunk Answers posts:

How to call a Python script from an HTML view?

dsollen had an HTML dashboard and wanted to …

» Continue reading

Box Plots: Making Custom Visualizations

This is the first of a two part series on implementing Box Plots in Splunk for security use cases.

Analyzing complex data is difficult, which is why people use Splunk. Sometimes patterns in data are not obvious, so it takes various ways of looking at aggregate reports and multiple charts to ascertain the important information buried in the data. A common tool in a data analyst’s arsenal is a box plot. A box plot, also called a box and whisker plot, is a visual method to quickly ascertain the variability and skew of data, as well as the median. For more about using and reading box plots, read the excellent and succinct post by Nathan Yau of the Flowing Data …

» Continue reading

Splunking Continuous REST Data

One of the ways vendors expose machine data is via REST. There are a couple of ways to get REST data into Splunk today:

  1. Use Damien Dallimore’s REST API Modular Input – you can provide a custom response handler for this input to persist state.
  2. Use the new Splunk Add-on Builder – this method will do a “one shot” of the REST endpoint – meaning, every time the input runs, it will get all the data every time.

In this post, I will show you how to implement a cursor mechanism (i.e. pick up where you left off last time) for REST endpoints that continually have new data over time using the checkpoint mechanism built into modular inputs.

The Data Source

For …

» Continue reading

Humanizing Security Data Visualization

Visualizing and displaying complex data is hard. Understanding complex data is harder. Rapidly making operational decisions based upon complex data is extremely hard.

Historically, operational security analysts rely on alerts, tables, and charts on dashboards or in email to pull potentially useful information out of the vast sea of data dumping into their analytic systems. This has always been problematic due to the combination of false positives and understanding the context of data filtered through the human brain. Most of the standard methodologies for displaying complex information make it harder, not easier, for humans to understand the information they seek in a timely and operationally useful manner.

Everyone has seen dashboards with a wall of text in tables interspersed with …

» Continue reading

What size should my Splunk license be?

This is a pretty common question in Splunkland. Maybe you’re an admin wondering how much license you’ll need to handle this new data source you have in mind for a great new use case. Or you’re a Splunker trying to answer this question for a customer. Or a partner doing the same. Given how often this comes up, I thought I’d put together an overview of all the ways you can approximate how big a license you need, based on a set of data sources. This post brings together the accumulated wisdom of many of my fellow sales engineers, so before I begin, I’d like to thank them all for the insights and code they shared so willingly. Thank you

» Continue reading