Splunking Outside The Box -v2.0

If you attended my technical presentation @ the Splunk 2010 users.conf event last called “Splunking Outside The Box“, then you’re probably aware of just how esoteric my thinking can be when it comes to creatively leveraging Splunk for the more non-sensible, yet highly educational use cases.

For example, I showed-off my Splunk for Texas Lotto App, which my team here @ Splunk uses each month to pick our “winning” numbers.

So far, we’ve won about $26…but we’ve spent ten times that amount along the way. But that’s beside the point.

Anyway, at this year’s conference I am hoping to avoid those everyday boring run-of-the-mill searches and get you thinking outside the box yet again.

Yes, I plan to …

» Continue reading

Splunk and Sports

As I write this, the NBA Basketball playoffs are under way and some of you may be interested in what is currently happening with the sport. If you are a Splunk user, it may be worthwhile to take a break while you send a search to the background and get the latest playoff news and standings from Splunk Web itself. To meet that objective, I’ve created a simple XML dashboard which shows the latest RSS headlines from NBA.com and ESPN in 2 different panels with their respective web pages in iFrames. In this manner, I can do my Splunk work and also jump to this basketball app to catch up with what is going on in the NBA. Here’s a …

» Continue reading

Correlating with Splunk IP Watchlist

Last year, fellow Splunker, Dave Croteau, created a prototype to daily index the world’s top 100 suspicious, or in some cases malicious IP addresses, by using a list created by the dshield.org web site. One thought is that these addresses may be compromised by trojans or botnets, so you would not want them to appear as sources connecting to your network. Dave also used the Splunk Maxmind add-on to show a simple dashboard that map’s these addresses to country and city with the Splunk Top command.

Next, I took this app and changed the scripted input to use curl to gather the data so the same approach could be ported to Windows as well as *Nix based machines. I also …

» Continue reading

Splunk and Astronomy, Part 2

I recently attended a series of fascinating short lectures, given in layman’s terms for the average audience, conducted by Astronomer Andy Green from Stardome (based in the UK). The topics that were covered included lunar landings, star formation, and planetary impacts. It is the last topic that I would like to discuss here. Andy presented a history of terrestrial and extra-terrestrial impacts for things like when an asteroid or a comet strikes a heavenly body. The most interesting heavenly body is, of course, the Earth itself. Today, there are a handful of organizations that track Near Earth Objects (NEO) and the log files that they collect can be indexed by Splunk. In the past I have already mentioned one use

» Continue reading

Consolidation with Splunk

One of the things I do regularly here at Splunk is talk to customers. Some are using Splunk for its full potential beyond search and indexing for in-depth analysis of their data with full use of the search language, comprehensive dashboards, and timely alerts. Then, there are some who use it for a single purpose such as monitoring logins, auditing operations, or general purpose troubleshooting. This, in itself, allows Splunk to show value for the task at hand. However, these same customers may also be using several tools and technologies for tasks which may be consolidated with Splunk. What follows is a discussion for using Splunk to monitor several aspects of a business, some by simply reusing the same indexed …

» Continue reading

Event Correlation

It has been a while since anyone has written a direct blog entry on event correlation here at Splunk so I thought I would write one today. Event correlation can loosely be defined as a technique to relate any number of events with some identifiable patterns (and optionally act upon the relationship). Security vendors may narrowly claim that event correlation is the ability to correlate security related events and alert upon their existence. This is a subset of what event correlation can be. For instance, in a hypothetical case,  I can correlate that if it rains on a major Monday holiday, end of day total sales are lower than average sales for a brick and mortar retail shop. This case …

» Continue reading

Weather Alerts in Splunk

Its been a couple of years since I first created the current weather conditions app that is on Splunkbase, so I decided to do something similar that is a little more timely. Current weather conditions are nice events to index as they give a time line for how things are going at a particular location and provide a basis for trend analysis. However, they do not provide insight into upcoming severe weather, which are more important events to track.

Fortunately, the weather underground provides a REST API to gather severe weather alerts using a zip code. I built a scripted input Python script to gather these alerts and the standard output of each call is indexed by Splunk. The …

» Continue reading

Astronomy and Summary Indexing

I had the pleasure last week of viewing Saturn’s rings at Rutgers University’s observatory. It was my first time actually seeing the rings through a professional telescope and the planet does look like what we often see in text book pictures. After the viewing, I started thinking that astronomy records a lot of data that needs to be indexed for search and aggregated for reports. I asked the professor conducting the tour if he had any logs for astrometry data and he took out his paper notebook to show it to me. Obviously, in Splunk terms, that was not what I was asking to see.

In seriousness, the professor told me that optical telescopes, radio telescopes, and spectrometers can generate …

» Continue reading

Universally Indexing Business Data

By the title of this entry, you may be thinking that there is some new capability within Splunk to index other types of data. That’s not the intention. From its roots, Splunk was used to index and search on IT data. It still is. However, because of the flexible nature of the software to index any type of time series text data, customers using Splunk do not restrict it to indexing only IT data. From the beginning Splunk was designed to universally index data from a variety of sources as long as the data was eventually ASCII text in representation.

Due to this inherent capability, Splunk can index data that is not necessarily meant for consumption by IT staff and …

» Continue reading