“Serendipity is looking in a haystack for a needle and discovering a farmer’s daughter
“ - Julius Comroe

I just read the quote in a presentation from Matt Jones of BERG at the DXf conference. There is so much i love about this presentation i don’t know where to start. Just click through it ( embedded below ) and have your own reaction. It’s clearly designed to be a fun/light read. I think I clicked at about one slide per few seconds. Then went back and stopped on a few that really spoke to me. It was entertainment that made me think which then made me smile.

At its heart, splunk is a time machine. It allows someone to go back in time and “see” what their world looked like at any given moment and to look for trends, anomalies, volume, momentum, etc. If you put enough data into splunk, you can re-live the past with microscope, hit “play”, slow it down, speed it up, draw it on a chart, compare it to another time. We are all about time.

Every week i run into someone that is having performance issues and they are not aware you can just add another server or two or ten. I’ll travel to meet a company and I’ll ask how many servers they are using for Splunk to search/index/report on a terabyte a day. They will say a couple. I’ll then ask how many they have for a similar sized hadoop or data warehouse project. They will say 50 to 100X that number. Look if your going to give these systems 300+ servers, can we please get 15?

Somehow there is a breakdown in our communication that we scale like all other good architectures.

The following are hopefully some easy pictures to help tell the story. It should be extremely simple and straight forward, to the point of being obvious - if not bug me and i’ll try again.

I was in a meeting last thursday where some “important-people” ( not sure if they want to be named ) dropped the D word ( “disruptive” ) several times. They were presenting a slide that proved-out an age-old (1994?) adage that the key to success is ( can be ) a disruptive business model. It’s one thing for professor Christensen to talk about it, and another when its bankers have a slide for it. Personally I need to be reminded of its importance every day, since being disruptive was one of the most important guiding principals when founding Splunk. As we grow, and become more established, i hope we continue to be a disruptive leader - it certainally faces constant tension.

Hearing the D-word reminded me i wanted to post about Steve Jurvetson recent video of a talk at Stanford. I can’t tell if the talk was great, or it just spoke to me as i just needed a entrepreneurial boost. Also, I find myself very much into nano/bio engineering these days as it seems the next wave of innovation and DFJ is clearly backing some of the most innovative work. The great thing about Splunk is trying to make heads-or-tails of very large data and the more time I spend with it, the bio/nano space the next frontier.

How people use Splunk is often a surprise to us - at least they are going beyond our original intent. Initially we thought of splunk as a search engine for log files, Google for your logs if you will, to help IT folks troubleshoot their complex systems. Quickly we found that users started Splunking config files, network packets, source code, email, etc. Over the years our customers have been dragging us into all sorts of new uses-cases like global windmill power plant data analysis, protein structure prediction, or just something simple like analyzing user behavior on a website.

Lately we have started to see the collision of Splunk and big data analytics, usually with hadoop based tools, vertica, aster, greenplum, etc. In most cases there is complimentary value with these guys as they are better at some things than splunk, but there are use-cases where splunk by itself is just fine. Either way, Splunk is getting dragged into the big data area since we often are the collectors and often the primary indexer of long term historical data.

It was interesting to see Curt Monash, veteran database analyst and guru, post about splunk. If was a very short introduction to Splunk, but our appearance on his list signals our entry into a larger big data discussion.

Last week Luke Kaines, The Master of Puppet, held a very well attended Puppet Camp here in SF. He drew a fantastic attendance from top notch companies - I was most impressed with the technical quality of the presentations and breakout sessions ( quality food too! ). These types of events can often be mundane or boring - this was not. Kudos to Luke for building a quality community.

I had the pleasure of meeting Luke some three years ago back at a BayLISA event where I saw him win over a tough audience with an early incarnation of Puppet. Its been fun watching him over the years deliver on that early promise and for continuing to win over a very tough crowd.

Recently I’ve been polling our customers how they do configuration/change management. Interestingly, I have noticed people mostly fall into two camps:

• A very large percentage that use Puppet
• A equally large percentage use nothing or home grown

It caught me off guard that such a large number use Puppet and equally surprised that there was no #2 vendor solution. Great news for Luke and team.