Please Bypass the Database

It has been a while since I posted to these pages and I am sure there may be one or two of you who misses my erudite musings or as some may say ramblings of a longtime Splunker. Either way, here’s my first post for 2015.

I have noticed that there are a quite a few deployments in the world that write time series data to a log rotated file and have another process translate those events into a rows and columns to be ingested into a relational database. After this extract, translate, and load process (ETL), they then use SQL to gather their database records either add-hoc search or for aggregate reporting. This practice has been going on for …

» Continue reading

Shazam – real-time business analytics with Splunk

shazam_logo2Business analytics and data visualization are hot topics right now. One of the questions I get asked a lot is “so what’s different about Splunk when it comes to business analytics?” The short answer is the kinds of data we can analyse, the amount of information we can manage, the power of our search language and the real-time nature of the data. We’ve got some great examples of customers such as John Lewis, UniCredit, Domino’s Pizza, SurveyMonkey, Comcast etc. using Splunk for business analytics. I’m delighted that we can now add Shazam to that list:

 

 

Shazam Drives Improved User Engagement and Increased Revenue with Splunk Enterprise (press release)

Splunk and Shazam. Real-Time Insight Drives

» Continue reading

Valentine’s Day 2015 – a Splunk data-driven guide to finding love

ILoveYouBinaryFirstly, happy Valentine’s Day. Last year I wrote a slightly dubious love story to big data that you can read and cringe at here. This year I thought I’d try and be a bit less cheesy romantic novel and be a bit more loved up data analyst. I’ve pulled together a few different dating, gift and Valentine’s spending data sets and indexed them in Splunk. The outcome is some interesting analytics and insights which I’ll try and summarise into some love-based Operational Intelligence (sounds romantic doesn’t it?) that will give you a sure fire, data driven way to have a successful Valentine’s Day. Please consider that last time I tried to predict something with data (the winner of the …

» Continue reading

New Business Insights from Data Mashups

Over the past few months, I have been talking to a number of customer and prospects on how they use Splunk.  It is fascinating how customers that started with a use of Splunk to monitor website or application have expanded their use of Splunk for Business Analytics.  In this blog post, I wanted to share some ideas and customer examples for using Splunk for Business Analytics.

 

Many of the customers start using Splunk for monitoring their online properties – website, mobile apps or applications  for uptime, response time, speed etc. by indexing weblogs or application logs.  These logs contain critical information that can go beyond monitoring the online properties. User activity, application usage, e-commerce transactions leave digital breadcrumbs that …
» Continue reading

Christmas 2020. Will big data and IOT change things for Father Christmas? Part II

ChristmasTreeBinary3

 

 

 

In part 1 we discussed how Father Christmas is planning to use sensor data for the Internet of Toys.

 

In part 2 we’re going to discuss how he is going to use very large data sets to build out his Christmas 2020 technology strategy.

 

 

 

 

 

Big Data & Analytics

There’s a lot of information that goes into making Christmas a success. This data includes:

  • Social media sentiment about good or naughty children
  • Christmas present lists from children (both digitized scanned letters and increasingly electronic present lists)
  • Data from toys and manufacturing equipment to spot patterns in quality control
  • 500 years of Christmas Eve delivery data to help optimize sleigh route planning
  • Reindeer biometric information to ensure optimum
» Continue reading

Christmas 2020. Will big data and IOT change things for Father Christmas? Part I

Data TreeAfter last year’s case study on Santa Claus International, I recently had the opportunity to spend some more time with Father Christmas discussing his long term plans and business strategy for the next 5-10 years called “2020 Vision – Noel Limits”. We covered how the “advent” (geddit?) of technology over the last couple of years has changed the way Father Christmas has to prepare and deliver “positive festive outcomes” throughout the year. We talked about everything from The Internet Of Toys, use of big data and Ho-ho-hodoop, the unfortunately named Christmas Retail Analytics Platform and Augmented Sleigh Service. In part 1 we’ll review Father Christmas’ plans for The Internet Of Toys and in part 2 (tomorrow) we will cover …

» Continue reading

Protocol Data Inputs

It must have been about a year ago now that I was talking with a Data Scientist at a Splunk Live event about some of the quite advanced use cases he was trying to achieve with Splunk. That conversation seeded some ideas in my mind , they fermented for a while as I toyed with designs , and over the last couple of months I’ve chipped away at creating a new Splunk App , Protocol Data Inputs (PDI).

So what is this all about ? Well to put it quite simply , it is a Modular Input for receiving data via a number of different protocols, with some pretty cool bells and whistles.

pdi

 

So let’s break down some of …

» Continue reading

Detecting outages caused by unauthorized changes

Splunk is a great solution to search, investigate as well as monitor your IT environment, whether it is application, infrastructure or network related. One perplexing issue to detect is related to unauthorized changes. Per ITIL, an unauthorized change is a “change made to the IT infrastructure that violates defined and agreed Change policies”.

Let’s take a simple example where you have a multi-tier application and one of the admins made a change on one of the configuration files without running through the CAB or the Change and Release manager for impact analysis. This config change resulted in an application outage. Using Splunk, you can easily detect the outage,  no doubt about that.

The challenge is how can you isolate the …

» Continue reading

.conf 2014: The Community Report

Whew! Welcome back from .conf, everyone. I know it’s been two weeks since we all hung out together in the Community Lounge, but it still feels like we only just left the MGM yesterday…

All for you: the Community Lounge

This year at .conf, we created an intentional space for our amazing user community: you. You folks are the reason we’re here, and we wanted you to have a cool place to meet other Splunk users, talk about the stuff that matters to you, and get a little fun in at the same time.

The Answers Desk

The Splunk Answers Desk was, as always, hugely popular. Our staff of lab-coated (and sometimes be-fezzed) Support and Sustaining Engineering specialists were kept busy helping …

» Continue reading

From big data to a 360 degree customer view with Hunk and Hortonworks

Age of the customerYou can’t really escape the fact that we’re in the age of the customer. From CRM to the “long tail” to multi-channel to social media brand sentiment to Net Promoter Scores – it is all about customer experience. Big Data has an important part to play – no great revelation there but how do you actually do it? There are an awful lot of questions that come up when it comes to Big Data and customer view;

What should my architecture be? How do I put together the right data strategy for the short and long term? How do I get the value from the data? How do I build customer analytics on top of my data? How do I …

» Continue reading