I love “log management.” I hate log management.

I love log management because years ago it was the impetus for IT to move beyond simple SNMP monitoring to collecting and trying to understand a much richer set of data about complex environments.

I hate log management for over the years it has been co-opted by vendors and analysts who’ve pigeon holed it into yet another IT management silo. These vendors and analysts have narrowly defined log management as the collection and storage of logs in some locked repository used to generate static reports to satisfy regulators, auditors and IT governance boards.

Why am I so bitter?

First it turns out logs are critical to many other stakeholders in the enterprise. Operations needs real time access to logs in order to find and fix problems and improve mean time to recovery (MTTR). Security needs logs to catch bad guys. Business people need logs to understand customer and service behavior and provide service level measurements. So locking up logs in a static repository designed for one constituency severely limits their value and diminishes the return on investment not only in a log management solution but also the return on your IT assets overall.

Recently Matt Asay wrote a thoughtful piece about how some technology companies are consumerizing the computing experience. In the case of Apple, Business Week writer Peter Burrows has also recently wrote about The Mac in the Gray Flannel Suite exploring how CIOs are testing the appetite for Macs in the enterprise. Michele Goins CIO at Juniper Networks recently ran a test among the company’s 6,000 employees discovering that 25% wanted a Mac.

Consumerism of the enterprise computing experience is well underway with Apple, Google, SalesForce and even Cisco’s TelePresence and WebEx offerings. According to Matt, all of these products delight users with a positive user experience by focusing on adoption first and dollars second. “Simple, fast and useful,” is the key.

Could it be that the consumerization of IT is far behind? How many enterprise management vendors focus on adoption first and dollars second? Can you honestly say that any of your vendors put you and your users first? Do the words “simple, fast and useful” come to mind as you’re writing the check for your maintenance renewal every year?

This week we were rolling in Las Vegas with Interop at one end of the strip and the Microsoft Management Summit at the other end. At Interop we launched the Splunk for Change Management app. And at MMS the Splunk for Windows Management app made it’s debut. Both apps make use of the Splunk Platform which provides a common set of services and APIs making it easy to create and integrate applications that leverage vast amounts of IT data. These are the second and third applications in a series of new releases we’ll be doing this year. Splunk for PCI was the first app launched last quarter.

Splunk for Change Management App

Splunk for Change Management takes advantage of the fact that we index not just logs but configurations and file system changes as well. It also leverages a little known (but I think soon to be much more popular) Splunk search command called diff. Diff lets you easily compare two search results and returns a single result that is the different between the two. You can compare values of specific fields of results as well as every line of multi line events and files. This makes it really easy to compare configurations across lots of locations. Splunk for Change Management leverages these capabilities and brings integrated change audit, change detection and change validation.

This week we’re at FOSE 2008 demonstrating how we’re collaborating with US Federal Agencies. A number of agencies have already joined the Splunk community including:

Many of these customers are applying Splunk to extreme applications with large data volumes from many different disparate sources. As you can imagine the complexity of security and compliance concerns, agency interactions and a sophisticated web of outsourcing to federal system integrators provides fertile ground for IT Search as a new way of solving all kinds of problems.

Typically our collaboration involves operations, security and compliance people from both the agency and system integrator sides. Agencies continue with their pursuit to cut costs and outsource while being driven with a host of new projects every year. And system integrators continue to search for new ways to bid more competitively by demonstrating new ways to more efficiently develop, deploy and manage technology. This means the business of managing our nations IT infrastructure is significantly more complex and dynamic than ever.

Without a doubt the past week has been the most amazing week in Splunk history. The crazy coast to coast multi-city launch left us all exhausted and electrified. A few of the things that stick in my mind…

First Splunk 3.2 including Splunk for Windows went live on our download page last Saturday and more than 40% of our downloads in the past week have been for our new Windows version. Then Nick Selby of 451 Group wrote an analyst brief on us. He said, “Splunk is awesome: it’s multiplatform, easy to install and easy to use. And with an abstraction layer of logs, configuration files and system messages, traps and alerts, it’s seriously useful.” 451 has a reputation for ripping vendors, so we’re flattered.

Dana Gardner, analyst with Interarbor wrote a very eloquent analysis of our platform launch on ZD Net. “Splunk has created the means to offer developers easy access to that data and the powerful inferences gleaned from comprehensive IT search. That means the data can go places no log file has gone before,” says Dana. Developers are certainly doing some way cool things with Splunk.

Recently, Johnvey Hwang wrote a post called Standing on Our Own Platform. He was the first one at Splunk to break the ice and use the “P” word. Now it’s out there. What do we see when we stand on our own platform? While only you and the future will tell us — there are a few things we hope to see on the horizon.

First, it’s our belief there’s a lot of money out there wasted on point products for managing networks, servers, applications … even security. A lot of these systems redundantly collect, transmit and store much of the same machine generated data. Think of the network, storage and administration resources duplicated on all this stuff. By providing a platform where the same IT data can be managed once, resources can be freed for other projects.

Second, none of these products work together. If you’re running a network manager to collect and look at SNMP and netflow data you know it doesn’t integrate with your log management system and of course neither talks to your SIEM, SOA, virtualization or application framework monitoring consoles. Building a dense index of data from all of these tools enables correlation across all your silos of instrumentation.

The US economy is heading into a recession and technology spending is in for a steep decline in 2008. So every major prognosticator and news outlet from the Wall Street Journal to the Financial Times would have us believe.

Are these people watching the same movie I am? There are two problems I have with this economic hyperbole. Yes that’s what it is. I guess it sells newspapers and gets people to watch things like CNBC. But boy is it misleading.

First of all, in macroeconomics, a recession is a decline in any country’s gross domestic product (GDP), or negative real economic growth, for two or more successive quarters of a year. Yet nobody that I’ve read is forecasting negative growth. They’re forecasting a potential slow down in growth from the current 3.5% per quarter to 1.5 to 2.5% per quarter. But the news outlets feel compelled to use the “R” word just to get attention. Totally irresponsible.

On to my second gripe. With regards to technology and IT spending, I believe, based on what I see, we are in beginning of a long-term gradual increase in IT spending within large enterprises that started eighteen to twenty four months ago.

I’ve written previously about our experience this year raising a $25M Series C round of venture financing. Venture Diaries: Part One discusses why you want to think before you act and investigate who to target as potential investor partners. Venture Diaries: Part Two looks at how to perform your investigation. In this third part, I look at how to handle the horse race that inevitably develops once you get a few term sheets.

For me it all started when the first term sheet came in. Funny how some VCs still use fax machines. I had to go figure out where ours was. In the current seller’s environment (yes that’s what you are, a seller of equity in your company) one thing to keep in mind is your first term sheet will just be a starting point. Expect that it will probably be lower (perhaps significantly lower) than where you want to end up. Also expect once the first term sheet comes in things will really start to heat up. Nobody wants to miss out on a good investment and VCs are just egotistical enough to really help your cause. However, you should realize each VC has their own style. Some will try to move first in hopes of stealing the deal from others. Others will try to wait till the end and trump any offer — figuring the last hand in has the best chance.

According the National Venture Capital Association (NVCA), there are 798 venture capital firms managing more than $235B in the United States. These are long-term, professional investors who specialize in funding and building new, innovative companies.

So how do you figure out who to approach for funding? This is the area where I find entrepreneurs make the biggest mistakes. Most of us approach investors we know. Perhaps you have a friend who knows a VC or you have a friend who is a VC. How do you know if your friend or the person you get introduced to is the right investor for you? Most likely they’re not. Not all VCs are alike. Some are geared for early stage and some are not. Some are suited for late stage investments while others just say they are.

You can’t always trust what an investor says their appetite is either. I’ve pitched to investors who say, “yeah we do Series A” only to be barraged by questions like, “how many paying customers do you have that we can talk to.” On the other hand, I’ve presented to wanna be later stage investors that were only prepared to pay an early stage price.

Last week I was in NYC for Interop 2007. Interop in NY is a significantly smaller conference than the big brother Interop in Vegas. I’d say there were 7,500 to 8,000 people at Interop NYC this year, compared to 18,500 in Vegas back in May. Somehow though I always find the New York show more interesting. Perhaps it’s the lack of constant firefighting in the NOC that gives us all more time to have meaningful conversations about the latest networking technologies. Plus somehow New York just seems to have more substance than Vegas. Call me crazy but…

This was also the first Interop where we had a chance to apply the magic of Splunk genre 3.0. We had a record number of searches in the NOC (despite the smaller show). I’m not surprised. 3.0 is so cool the way it automatically extracts fields out of data streams from all kinds of networking gear.

Now there are lots of people who know more about networking and security than I do, but here’s a simple investigation I did with Splunk.

1. I started with a simple search for “failed password.” This picks up firewall and router hacking attempts (typically ssh) sent to Splunk using syslog forwarding.