Dashboard Digest Series – Episode 4 – NFL Predictions

In Episode 4 we will take a look at the four downs of football. We used the Machine Learning Toolkit and more than a decade of NFL data to build models to make predictions during NFL games.

In order to make it quick and easy to plug in a scenario and visualize the most likely outcomes, we made a simple dashboard so editors at Sports Illustrated could try it out during a game. You may have seen the dashboard if you were watching CNN before the Super Bowl earlier this year:

Purpose: Predict the next play
Splunk Version: Splunk 6.4
Data Sources: Every NFL play and player since 1999
Apps: Machine Learning Toolkit, Shapester

The data contains a lot of fields

» Continue reading

Dashboard Digest Series – Episode 3

energy_small

Welcome to Episode 3 of the Dashboard Digest series! At Splunk we love to eat our own dogfood so in this episode we will see a dashboard showing energy and water usage at Splunk headquarters in San Francisco! Additionally you’ll see a few new custom visualizations that became available for use in Splunk 6.4 as well as use of the Machine Learning Toolkit.

Purpose: Display and analyze building energy and water usage. Use machine learning to forecast energy usage, detect outliers and look for anomalies.
Splunk Version: Splunk 6.4 and above
Data Sources: Sensor data in JSON format coming from Aquicore devices.
Apps: Machine Learning Toolkit, Water Gauge Visualization, Calendar Heatmap Visualization

Summary of tips/tricks used:…

» Continue reading

Dashboard Digest Series – Episode 2: Part Deux

geoheatmap_hurricaneBefore moving on to the next episode 3 I decided to do a part two of Episode 2 – Waves!  The reason being is two-fold.  1) Splunk Enterprise 6.5 was recently released and 2) Hurricane Matthew had quite the effect on some of these buoys/stations.  See the original blog post here: Dashboard Digest Series – Episode 2

Purpose: Display meaningful statistics on NDBC buoy information in historical and real-time.  Easily drilldown, aggregate and visualize data from 1000s of buoys transmitting information.
Splunk Version: Splunk 6.5 and above for table coloring
Data Sources: Polling NDBC RSS feed that produces JSON payload
Apps: Add-on for NDBC, Custom Cluster Map Visualization, Clustered Single Value Map Visualization, …

» Continue reading

My Splunk Origin Story

A World Without Splunk

In my pre-Splunk days, I spent significant time leading the vision for standards and automation in our company’s large distributed IBM WebSphere Network Deployment environment. Even though we used standard build tools and a mature change process, significant entropy and deviations were introduced into the environment as a product of requirements for tuning, business, infrastructure, security, and compliance.

As a result, we were unable to recognize the scope of impact when it came to security vulnerabilities or violations with 3rd party compliance. Even worse for us, we spent way too many staff-hours trying to replicate issues between production and quality assurance environments because we had no easy way to recognize the contributing configuration differences.

It’s a Bird, It’s a

» Continue reading

How’s my driving?

It was the summer of 2014. I was well into my big data addiction thanks to Splunk. I was looking for a fix anywhere: Splunk my home? Splunk my computer usage? Splunk my health? There were so many data points out there for me to Splunk but none of them would payoff like Splunking my driving…

Rocky Road

At the time, my commute was rough. Roads with drastically changing speeds, backups at hills and merges, and ultimately way more stop and go than I could stomach. But how bad was my commute? Was I having as bad an impact on the environment as I feared? Was my fuel efficiency much worse than my quiet cruise-controlled trips between New York and Boston? …

» Continue reading

NBA Finals 2015

I recently posted a blog about Splunking my golf swing and afterwards a co-worker asked if I could Splunk the NBA finals. He gave me some NBA data and while on a flight today I decided to look into the data a little with Splunk. I don’t know very much about basketball and you all probably have way better questions to ask of the data; nevertheless I gave it a shot on my flight. Note: CLE=Cleveland and GSW=Golden State Warriors

Each file had the date of the game and who played where as the filename.
[2015-05-22]-0041400302-CLE@ATL.csv
[2015-05-23]-0041400313-GSW@HOU.csv
[2015-05-24]-0041400303-ATL@CLE.csv
[2015-05-25]-0041400314-GSW@HOU.csv
[2015-05-26]-0041400304-ATL@CLE.csv
[2015-05-27]-0041400315-HOU@GSW.csv
[2015-06-04]-0041400401-CLE@GSW.csv

Since it was csv I imported it as such and set timestamp based on the date and “elapsed”.…

» Continue reading

All aboard with Infrastructure 4.0 — Splunk wins Deutsche Bahn Internet of Things Hackathon

Deutsche Bahn (DB) describes itself as the second largest transport company in the world and is the largest railway and infrastructure operator in Europe. With the popularity of Industry 4.0 and IoT in Germany, DB recently ran a “Deutsche Bahn goes 4.0” Hackathon over the weekend of May 8-9 2015. The concept was “We provide the data, you innovate with it”. Splunk participated with a crack team of two people, a copy of Splunk Enterprise running on a laptop and got their hands dirty digging into a labyrinth of infrastructure data. The challenge was tough: starting at 5pm we had 24 hours straight to analyze the data and demonstrate the value from it. After the final presentation of …

» Continue reading

Make it flash! Make it flash!

Splunk Traffic Lights

Splunk ships with some really neat visualisation options. From bar charts to gauges. Though sometimes they just don’t fit your requirements.

Wether that be something as simple as an custom icon or a super-slick D3 visualisation, Splunk’s framework makes it really easy to display your data in many number of ways.

One of the things I get asked a lot is: “Can we have a traffic light?”. The answer – yes! Let me show you how to light Splunk up in this post.…

» Continue reading

Mission Critical Analytics – Everywhere

You are so incredibly awesome.

That’s what I wanted to call the new release of Splunk Enterprise announced today, but instead we went with Splunk Enterprise 6.1. What this release represents is pretty remarkable. Why? Because it’s defined by what we see you do.

You’ve made machine data a valuable asset in your organization. Whether you’re in IT, on the applications team or on the security team, you’ve helped yield powerful results for the company you work for. So valuable that machine data insights are now more mission critical than ever.

You’re changing how you analyze data—you’re comfortable exploring and analyzing data, knowing that Splunk software will eat just about anything you throw at it and give you answers …

» Continue reading

Custom Icons in Splunk 6 Tables

“Daddy. DADDY! We’re out of Sriracha. Does Costco sell Sriracha? Can you go get some before you start working today?”

That was my five-year-old son at breakfast this morning, after he turned the Sriracha bottle upside down and banged the heck out of the bottom of the rooster-adorned bottle with his tiny fist, trying to get the last bits of the dark-red chili sauce deposited onto his scrambled eggs.

While I’m certain we will solve the 2014 Sriracha Crisis at the Brodsky household, the whole episode reminded me of a question (stick with me, you’ll see why) that a Splunk customer asked me a few months ago, which went something like this:

“When creating a dashboard in Splunk 6,

» Continue reading