Life, Liberty and the Pursuit of Log-i-ness

Technological democratization (say that 3 times fast – that’s one), the power of technology to allow the budget-constrained rest-of us to do things only well funded professionals could do before, is everywhere. And, if you give me a few minutes of your time, I’ll tell you how it’s even coming to the smallest of IT organizations.

So what got me thinking about technological democratization (that’s two) was that famous opening scene in Saving Private Ryan in which Stephen Spielberg showed an incredibly realistic re-enactment of D-Day and the Allied landing on Omaha Beach. Back in 1998 BD (“before Disney”) that scene cost $12 million to film (or 18% of the cost of the entire movie) and involved over 1,500 extras and tons of highly paid professionals at Industrial Light and Magic (ILM). The work was so specialized that they actually had one person whose job title (printed on their business card and everything) was “manager – underwater ballistic effects.” No, seriously, I met her.

Now flash forward 10 years to 2008 (still BD for ILM but not so much for Pixar) in which that scene is re-enacted by 3 graphic designers using only 2 military uniforms, 2 fake rifles, some rope, a (really BIG) camera and 1 station wagon. Total time to produce this knock-off masterpiece? 4 days. Although the budget was never shared, just looking at the station wagon (and, let’s face it, the graphic designers) makes it clear that it was WAY below $12 million.

How did they do it? Technology of course. In the intervening 10 years the technology had improved such that you didn’t need all the specialized hardware and personnel to do almost the same work. In short, technology has democratized the process of elaborate filmmaking such that it could be done with less people, less costumes, less money and less time (OK, maybe more Red Bull).

This trend of “technological democratization” (that’s three) is everywhere and you all know the stories. Industries such as music and publishing (along with associated distribution work) have all seen their costs lowered dramatically while the number of producers/stars has gone up just as dramatically. Ask your kids their favorite television channel and they will probably say “Television? Oh you mean YouTube.” (and chances are they are watching a mysterious woman unwrap Disney toys – as she is YouTube’s highest paid star and made $5 million last year doing that.)

Grommet-Comic-03012016-blog How is this democratization helping you solve your IT, Sys Admin and Dev Ops problems in 2016 AD (“after Disney”)? Well just as above it’s all about finding technology that allows you to do the work of 1,500 extras (or employees) when all you have are 3 people and a station wagon (have patience, you’ll get that Tesla eventually).

So since you are here you’ve probably already figured out that the equivalent of iMovie or Adobe Premier for IT is automated log search and analysis. But what should you look for in terms of the features of your log search and analysis solution, as they all aren’t created equally? In terms of technology democratization, think of these as your “rights.” As an IT problem solver you have the right to:

  • Automatic pattern recognition – So much of the work in solving a problem is detecting the root cause, and detecting the root cause involves spotting either a pattern OR a break in the normal pattern. If you have software that can speed that process you can speed time to resolution
  • Sophisticated Visualizations – While we dream of a system that can automatically detect key patterns we know that is a long way off. In the interim we still have to rely on our own eyes and experience, so it is important to make sure rich enough visualizations are fed into our brains (or to use my own rich visualization – “a computer made out of meat”)
  • No coding overhead – Many tools designed to help reduce the number of “extras” you need to have should say right on the label “some assembly required, batteries not included”. You are out to buy a solution, not a development project, because you’ll need more bodies for that. When I buy a hammer I don’t want a hunk of iron and a portable smelter, I want a hammer. Any typing I do should be to build a query to answer my question and even then it would be nice if the queries sometimes get written for me
  • A complete set of deployment options – Some of you do classic “on-prem” IT, some of you do “cloud.” Whatever your IT “lifestyle choice” is, you should be able to get the same benefit (can you tell we’re headquartered in San Francisco?)

So when looking at a solution for log search and analysis don’t forget your rights. The good news is thanks to technological democratization (that’s four, now I’m just showin’ off) you can now exercise your rights in a package you can afford. Seriously you need like $3 a day* lying around to get that capability from Splunk. And with any leftover budget I know where you can get a former manager of underwater ballistic effects on the cheap.

Marc Itzkowitz
Director of Product Marketing
Splunk Light

The $3 per day price is based on an annual license fee of US $900 for indexing up to 1 gigabyte of data per day using Splunk Light Software, and an annual subscription fee of US $1080 for indexing up to 1 gigabyte of data per day using Splunk Light Cloud Service.