Running two Universal Forwarders on Windows

We get quite a few requests on how to run two Splunk Universal Forwarders on the same Windows host. Why would you do this? The primary reason is that you have a lab environment and want to compare one version of Splunk to another during an evaluation of a new version. You may also have two sets of files you need to ingest into Splunk and the files have differing access permissions such that Splunk needs to run as different users. It’s really an edge case and definitely not something you want to generally do in production.

In Linux, this is a fairly simple process – just install to a different directory and change the ports and you are done. …

» Continue reading

The 2014 CyberPatriot National Finals

This blog post was jointly written by Tolga Tohumcu and Bert Hayes… Tolga mentored the student teams before the completion, Bert was on-site at the competition to help out in person, and Enoch Long was working behind the scenes to build relationships with folks running the competition.


The 2014 CyberPatriot National Finals took place recently at the Gaylord National Resort and Conference Center in National Harbor, Maryland with all of the spectator appeal of a competitive archeological dig.  Two shifts of high school aged students made up a total of twenty eight different “Blue Teams” and tested their mettle by defending their networks from a pack of active, aggressive, and skilled attackers (the Red Team). The CyberPatriot program …

» Continue reading

Cisco Security Suite 3.0.1 – Now with ISE

The Cisco Security Suite was recently updated to work with Splunk 6.  As mentioned in the previous release, one release is not enough to get all the Cisco security related information integrated into the suite.  With version 3.0.1 of the Cisco Security Suite, Cisco Identity Services Engine (ISE) has been added.  Over 20 ISE-related dashboards have been integrated into the suite.

Cisco with ISE



ISE is really powerful and adds a lot of additional data that can be correlated.  For instance, say you have an IP address from somewhere in your environment.  ISE can tell you which user is using that IP, what type of device the user is using, the posture of the device, and much more.  Therefore, in …

» Continue reading

Masters of Machines – Operational Intelligence in EMEA


t800Firstly – apologies – this isn’t going to be a blog post about how Skynet wins and Terminators take over the earth. It also won’t be about how Keanu Reeves wearing cool sunglasses(AKA Neo) saves us, via The Matrix, from sentient robot squids. Splunk recently commissioned Quocirca to conduct a piece of market research about the state of machine data adoption in EMEA and where organisations are in getting Operational Intelligence from that data.

We have a webinar this Thursday, 3rd April at 10am UK time (11 CET) and we’d love to have you come along and hear Bob Tarzey, the Quocirca analyst who conducted the research go through the findings.

You can get a copy of the …

» Continue reading

Search Command> stats, eventstats and streamstats

Getting started with stats, eventstats and streamstats

When I first joined Splunk, like many newbies I needed direction on where to start. Someone gave me some excellent advice:

“Learn the stats and eval commands.”

Putting eval aside for another blog post, let’s examine the stats command. It never ceases to amaze me how many Splunkers are stuck in the “super grep” stage. They just use Splunk to search (happily I might add) for keywords and phrases over many sources of machine data. Hopefully this will help advance some folks beyond “super grep” as well as assist those who may be new to Splunk.

When you dive into Splunk’s excellent documentation, you will find that the stats command has a couple …

» Continue reading

National Corndog Day 2014

This is the first annual blog post regarding Splunk HQ’s third consecutive year of celebrating National Corndog Day one day early.

NCD is a fake holiday, invented by children, sponsored by giant food and beverage corporations, devoted to the over-consumption of corndogs, tater tots, and exceedingly cheap light beer, and the viewing of the NCAA March Madness college basketball tournament.

The goal of NCD is to achieve the triple-double, i.e. to consume 10 corndogs, 100 tater tots (10 units of 10) and 10 beers between the opening whistle of the first and the ending buzzer of the last basketball game played. Have we done this? No. But we’ll keep trying, because they keep sending us free corndogs.

I would like …

» Continue reading

What’s new in Microsoft Apps

Splunk is exhibiting at the Microsoft Exchange Conference this week. If you are in town, please stop by booth #805 in the Eastside to see us. To coincide with this conference, we are releasing a whole slew of new apps and add-ons. Here are some of the highlights:

The Splunk App for Microsoft Exchange has undergone a huge makeover and now includes complementary functionality from the Active Directory Domain Services and Windows realm. We can correlate across those three platforms to see new and unique things. Want to understand how a Windows update affected the performance of your Exchange hosts? Now you have the information available to you. Want to arrange the app panels in ways that are useful to …

» Continue reading

Time based load balancing – Part 2

This is a follow up to my earlier post on the forceTimebasedAutoLB setting for outputs.conf.

There was some discussion (read: prove it to me) on the IRC channel about how would this feature behave with multi-line events or double byte characters. Well, you will be glad to know it worked flawlessly.

My events are from a Japanese Windows instance:

Screen Shot 2014-03-24 at 5.48.31 PM

I sent over 500,000 events using the oneshot command from the UF.

Screen Shot 2014-03-24 at 5.49.33 PM

And it worked as expected.

Lastly, there was some talk about data munging. Meaning part of one event being incorrectly added to another event. This can happen when Splunk doesn’t break a multi-line event proper. In my test, I didn’t even setup a BREAK_ONLY_BEFORE or LINE_BREAKER rule on the …

» Continue reading

Using Splunk as a data store for developers

A number of years ago, I wrote a blog entry called Everybody Splunk with the Splunk SDK, which succinctly encouraged developers to put data into Splunk for their applications and then search on the indexed data to avoid doing sequential search on unstructured text. Since it’s been a while and I don’t expect people to memorize the dissertations of ancient history (to paraphrase Bob Dylan), I’ve decided to write about the topic again, but this time in more detail with explanations on how to proceed.

Why Splunk as a Data Store?

Some may proclaim that there are many no-sql like data stores out there already, so why use Splunk for an application data store? The answers point to simplicity, …

» Continue reading

Splunk’s New Web Framework, Volkswagen’s Data Lab, and the Internet of Things.

There are many incredible features in Splunk 6. Pivot, Data Models and integrated maps really stole the show at .conf2013. But I really have to give credit to our developer team in Seattle for the massive leap forward in user interface possibilities with the addition of the integrated web framework, which is included in Splunk 6 but is also available as an app download for Splunk 5.

In the midst of all that Splunk 6 excitement at .conf, I was introduced (at the Internet of Things pavilion) to the team at Volkswagen Data Lab, and had some great discussions with them about their interest in using Splunk as a  platform for the management, analysis, and visualization of data from …

» Continue reading