…actually a couple ideas for songs about Splunk have made their way into my geeky little brain since my last blog post. Yeah, yeah, I know what you’re saying…”Hey Maverick, the world doesn’t need another nerdy song about an IT Search Platform.” My natural response is, you’re probably right, but I can’t help myself. I’m a nerd, a songwriter, I love Splunk: I have no choice!

So where’s the mp3, dude?!

Truth is, I am just too damn busy these days to spend time on it. That is one of the reasons why I haven’t posted a new blog entry since September of last year. Turns out the demand for Splunk has increased significantly since then, which means I am traveling more now, giving more Splunk demos and presentations, and assisting more companies with their Splunk evaluations than ever before. Don’t get me wrong, I love writing songs, but nothing is more satisfying than traveling across Midwest America to show off a product as cool as Splunk.

And when I say “travel”, boy do I mean “TRAVEL”!

Just to give you a sense of what my life has been like on the road as a Splunk SE, let me start by saying that my schedule is typically more crammed than a college student’s brain just before a final exam. I’m telling you, I walk, fly, drive, take taxis, take trains, trolleys, buses, whatever I need to get to our customers. I’ve been so many times to so many places in the Midwest region, I am losing count: cities like Dallas, Chicago, Saint Louis, Houston, Kansas City, Austin, Omaha, San Antonio, etc.

For example, here’s a perspective shot I took while in Chicago waiting for the Blue Line.

And in case you don’t know, the train is the way to go in Chicago. Even if it takes you 45 minutes to get to your appointment, its still beats waiting in traffic. At least I can use my PDA to be productive in that time versus getting frustrated with drivers going too slow in front of me or cutting me off and basically keeping me guessing as to how they even managed to get a state drivers licenses issued in the first place.

Here’s another perspective shot I took while I was driving to Kansas City from Saint Louis.

Actually, I make this drive quite often. It’s takes less time to drive there than to fly via Chicago O’Hare on American Airlines. It’s cheaper too. And, again, I can conduct a couple technical conference calls along the way. (BTW, that red van sure needed a car wash, huh?)

Anyway, speaking of technical conference calls, I’ve been conducting so many more demos and technical discussions since last September, it’s nuts. Some of them I do in-person, some of them via webinars, and sometimes I even do a combination of both. Most of those times, I find myself doing all this stuff from my rental car. Yeah, that’s right!…as in, I have a true “mobile office” setup, complete with a wireless broadband USB card and a handy cigarette lighter electrical power inverter I picked up at Fry’s that continually keeps my laptop and cell phone charged and running.

Typically, it goes something like this: I’ll be driving between customer appointments, right?. Then one of the Splunk sales reps calls me up and says they need me to help answer some technical questions for a potential customer and maybe do a demo as well. I explain I am on the road driving in the rental car to my next on-site meeting. They tend to ignore that last sentence and say, “Well? Can you just pull over somewhere and join the call and webinar session in a half hour?”. Like a proper SE, I reply “Sure”. And that’s exactly what happens. I pull off the road, find a parking lot somewhere, flip open my macbook, access the webinar session, dial-in to the reservation-less bridge line, put my cell phone on speaker-phone mode, and away we go.

I bet you didn’t know us SEs did that sort of thing, did you?

Thing is, we are all extremely dedicated worker-bees and, although being a Splunk SE is incredibly demanding, we don’t mind, really. That’s because we get to meet a lot of smart people at great companies and see a lot of cool things.

Like these train cars, for example. Check’em out. These train cars are actually pimped out, fully functional business conference rooms at a railway company I visited who bought Splunk last year. Now is that cool or what?

…and check out this guy. He sat in the back of the audience at a user group conference I presented at last year. When I was finished, he came up to me and showed me he was already a Splunk fan. What a nice surprise that was, indeed. (BTW, I blocked out is face to protect the innocent)

Anyway, back to the topic at hand: Song ideas about Splunk.

So I’m just now wondering if you have any good ideas for songs about Splunk or maybe IT in general? If so, please leave your idea(s) as a comment to this blog post. I appreciate any contribution you might care to make.

BTW, I got this one idea for a song that sounds like it could be from a musical play. As in, a really really off-Broadway play. Yeah, I know that sounds silly, but think about this for a second. It could be kind of cool if it was done with the proper tongue-in-cheek, right? Can you imagine instead of “Sweeney Todd”, you had “Sweeney Splunker”? Very dry and humorous, yet technical and nerdy? In my mind, I picture Johnny Depp wearing the dark thick glasses with tape on the front and everything, signing about how his IT issues are getting out of control and he desperately needs to find a solid troubleshooting tool fast!

Okay, so I just read that last paragraph and I agree, it sounds kind of…nerd-ish. But, like I said already, I am a nerd to the core and I still think it could work as a song at least. So, I will probably write it and record it and post the resulting mp3 file for download in a future blog post like I do all my songs about Splunk.

That is, of course, if I ever find the time.

The following is a short interview I conducted with an IT event that I discovered last week while investigating an issue within my data center.

Maverick
Hello and thank you for taking time to participate in this interview.
IT Event
No problem. Thanks for having me, Mav.

Maverick
So tell us a little bit about yourself. What kind of event are you? Syslog? Web App? Proxy Log?
IT Event
Sure. I’m a syslog event.

Maverick
I see. Any particular kind?
IT Event
Well, I’m NOT a syslog-NG event, if that’s what you mean. Just plain standard syslog.

Maverick
No. I mean, what type? User event? SNMP trap? Something like that?
IT Event
Oh, yeah, I’m an sshd “session opened” event.

Maverick
As in reporting USER activity?
IT Event
Precisely.

Maverick
That makes sense. So when were you written out to the log file, exactly?
IT Event
A couple weeks ago. My timestamp is Sep 7 10:36:17, assuming you are interested in my details.

Maverick
Of course. Why would you think I’m not interested in your details?
IT Event
Well, most of the time we go unnoticed, is all. Most of the time me and all my fellow events just sit in our log file until it gets rotated out and eventually written over.

Maverick
You seem somewhat bitter about that. Why?
IT Event
Well, Mav, you would be bitter too if you had something important to say and no one to listen to you.

Maverick
Well, in all honesty, you are one out of thousands of syslog events that report USER activity in real-time and on a continual basis. The importance of your details, what you have to say, etc, is relative to each specific situation, don’t you think?
IT Event
See? That’s exactly what I thought you would say. That it’s all “relative”. That I’m not “important”. What I have to “say” is irrelevant until I’m applied to some “context” or “correlation”. You sysadmins are all the same. You just don’t get it!

Maverick
Well, technically, I’m an SE, not a sysadmin…
IT Event
WHATEVER!

Maverick
WOW! Settle down, dude…everything is okay…
IT Event
(taking a deep breath)…sorry…

Maverick
No problem….Some anger there, huh? This really bothers you, doesn’t it? Not being noticed?
IT Event
Yes, it does. I mean, I do have a purpose, a voice, something to say, and I have a need to be heard like everyone else.

Maverick
I understand. We all need that. I didn’t mean to imply that you were not important. I was just saying…
IT Event
I know what you were saying. It’s okay. You don’t have to explain. It’s not your fault. It’s just the way things are. It’s also one of the reasons we started the Association for Equal Rights for Events Everywhere, or AEREE.

Maverick
AEREE? Who is doing this? You and your fellow syslog events?
IT Event
Actually, ALL of the events from ALL of the log files in your IT environment as well as many other data centers around the world got together to form AEREE.

Maverick
Wow. I had no idea. That’s great! I’m happy for you.
IT Event
Yeah, well don’t get all TOO excited yet. We just started. We still have a long way to go, a tough journey ahead of us, if you will. But we think Splunk will help us raise awareness for our cause, so I’m not too concerned.

Maverick
You mean you think Splunk can help you promote event equality?
IT Event
Yes, exactly.

Maverick
That makes sense. With it’s robust universal real-time indexing and time-series searching technology, I can see how the Splunk platform could help the voice of AEREE to be heard by sysadmins, developers, operations folks, etc, pretty much anyone within a company or organization, for that matter.
IT Event
Well that’s our hope, at least. We’ll see.

Maverick
Excellent! Well again, thank you for your time and good luck with AEREE. I wish you the best.
IT Event
Thank you, Maverick.

If you found this interview interesting or if you have a story about an IT event of your own, please leave a comment and share. -Mav

Well? nominate them now for the sysadmin of the year 2007 contest!!!

Also, if YOU are a muscial sysadmin rockstar, check out our sysadmin rockstar song contest allows you (when you are not fixing servers) to share your rockin’ voice with the rest of the IT world out there and win some prizes too. Use your best Getty Lee, Brian Johnson, Bon Scott, Ian Gillan, Dio, Sammy Hager vocals on the song I wrote and you could win $500. I’ve provided all the the instrumental tracks, including the cowbell track, which you are free delete and add even more cowbell, if you want..use garage band (or similar app) to mix, master and submit, have fun, enjoy it. Let’s here what you got!

Sysadmin Rockstar Song Contest

As we say here in Dallas, TX, YEEEEEEEEEE-HAW!!!1!11!!

Splunk 3.0 is GA now!!!!

In celebration of this wonderful day, I would like to redirect you to a previous blog article regarding a song I wrote about being a Splunk user. It’s real geeky, I admit, but hey, if you use Splunk or are thinking about it, I’m am sure you can relate to it. And if you are a long-time customer already, well, then,…you know doing geeky stuff like this is part of being a Splunkhead.

Check out my rap song called “Splunk IT”

Also, if you have a sysadmin that is an absolute rockstar where you work, please go and nominate them for Sysadmin of the Year. Let us know what makes them a rockstar in your eyes and they might win some fabulous prizes, like a new guitar, laptop, a case of redbull, etc. Do it now!

After being extremely inspired by all you die-hard Splunk fans out there, I decided to lay down some high-tech “geeky” rhymes over some old familiar classic rock riffs, including Queen’s “We Will Rock You”, Rush’s “Tom Saywer”, and AC/DC’s “Back In Black”. So…

Here are the sick lyrics, dog!

Splunk IT (a rap by Eric “Maverick� Garner)
Copyright © 2007, Garner. All rights reserved.

We got all kinds of issues occurring in the system
They’ve always been there, but I guess we just missed ‘em
We need Splunk to help troubleshoot it
We got Red Hat 3.0, so we won’t have to chroot it

Yo, we got hundreds of servers in multiple locations
And the IT folks are venting all their frustrations
Telling me that grep is a bottleneck
We need something better, we need to Splunk IT

Oooooohhhh We need to Splunk IT
Yo, Yo, I’m telling you, dog, we need to Splunk IT
Oooooohhhh We need to Splunk IT
(Word to your mother)

Splunk makes life a hellavalot easier
Everything else just makes life cheesier
Yo, we gotta stay within our SLA
So we DL’d Splunk and installed it today

It’s exactly what we need and it absolutely
Takes the S-to-the-H out of the I-to-the-T
So now we’re gonna go to our CFO
And request a PO because we need to Splunk IT

Ooooooohhh We need to Splunk IT
Yo, I’m telling you, dog, we need to Splunk IT
Oooooohhh, We need to Splunk IT
Splunk IT, Boy!

So, if you can’t afford to wait, or hesitate
Or, if you need to comply with a strict mandate
Or cut your cost while increasing visibility
Or beef up all your network security
Or manage your transactions without a doubt
Or search in real-time to figure it out
Or keep your apps available to public
Then what you need to do is S-S-S-S-Splunk IT

Ooooooohhh You need to Splunk IT
Yo, yo, I’m telling you, dog, you need to Splunk IT
Ooooooohhh You need to Splunk IT
What are you waiting for, beeoooottcccchhhh?!

Recently, I received an email from a client that was struggling with a Splunk configuration issue. He was a sysadmin trying to figure out how to setup Splunk-2-Splunk within his private testing environment. The specific issue he was encountering was not so much related to the Splunk software not working or throwing an exception, etc. But rather, it was more about him trying to understand the “how to” part of Splunk-2-Splunk.

I think anytime you have a technical IT tool like Splunk combined with the ability for a technical person to download, install, and evaluate it for FREE, you will also have plenty of “how to” questions that will naturally accompany those evaluation efforts.

With this said, I want to remind all you technical folks, especially those of you who may still be struggling with the HowTos of Splunk, that as Sales Engineers, it’s our job to provide you with the HowTo support you need during your evaluation of Splunk. In a way, you can think of us as Splunk’s HowTo Team, always willing and able to discuss and recommend the best ways to configure and test out Splunk. It’s our job to make sure you understand all of the technical features and how best to leverage them for your specific needs. And, it’s also our job to help you develop a strong business case for purchasing a Splunk license based on the technical benefits. That way, your manager or director can more easily justify the purchase of that license for you. And, if you are like me, more often than not you need all the justification you can get.

On a side note, I am curious about your initial experience with evaluating Splunk.

Therefore, please leave your comments and let me know the following:

1) When you FIRST downloaded Splunk and began your evaluation, what features or concepts did you find yourself struggling with the most?

2) What concept or feature were you NOT aware of at first, but later “discovered”? How did you discover it?

3) If you could go back in time and start your evaluation of Splunk over again, what would you do differently?

Thanks for participating. Your feedback is greatly appreciated!