Splunk 3.2 introduces a publicly available Python client library that allows external developers to programmatically interact with Splunk by importing a few key modules.

The easiest way to get started with the client library is to get into Splunk’s Python environment. Locate your Splunk install directory (/opt/splunk by default), and start the python interactive shell that comes with Splunk:

# bin/splunk cmd python


This will launch the interactive Python prompt, which starts off looking like this:

Python 2.5.1 (r251:54863, Nov 18 2007, 16:13:41)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
>>>


Starting a search

Import the Splunk modules:

import splunk.auth
import splunk.search as se


If you have installed Splunk with the default settings, then your hostpath is https://localhost:8089. The client library knows this default, so you can authenticate directly by providing a username and password:

key = splunk.auth.getSessionKey('admin','changeme')


The getSessionKey method automatically caches the session key in the current interactive session, so you don’t have to pass it along to subsequent methods. In a production implementation, or if you are connecting to multiple servers, you’ll need to keep track of separate session keys.

XML is a great format for exchanging information because it balances readability, extensibility, and compatibility across heterogeneous environments. However, its flexibility is also a disadvantage because it is far too easy to create a proprietary XML schema, resulting in lots of custom code to interface with various systems. Lots of custom code leads to brittleness, and brittleness leads to frustration. The key to salvation lies in standardization.

Enter the Atom standard: a standards-track schema that defines a generic collection/item container format in XML. Most people equate Atom to an RSS competitor, which is true, but that only covers half of what it does. The Atom Publishing Protocol is a well-defined protocol for performing CRUD (Create, Read, Update, Delete) operations on items over HTTP. The Atom Syndication Format, which is the most commonly used portion, defines the XML schema used to deliver data during a Read operation. Atom was spearheaded by Sam Ruby, and is now back by people like Brad Fitzpatrick, Tim Bray, Jeremy Zawodny, Mark Pilgrim, and is heavily implemented by Google.

Splunk 3.2 is available for download! This release is one of our biggest so far, representing a tremendous amount of effort by our engineering team, and is a product that I’m proud to stand behind. As I mentioned in my last post about our push for the Splunk Platform, a central tenet is to make a compelling product that developers will not only understand, but also enjoy using. While Dr. LogLogic rambles on about how catering to developers sucks, we know that developers are a huge part of our user base (drop by the #splunk channel on EFNet sometime) and we will continue to make Splunk as flexible and extensible as possible.

With 3.2, we have begun moving some of Splunk’s core services over to a proper REST API. Now, for those of you who have already been using the REST API in 3.1, the new API in 3.2 and beyond is distinctly different, and is intended to replace any older versions. Therefore, the REST API of version 3.1 and before will now be referred to as the UI API, and the term “REST API” will refer to the new API that I’m covering in this post.

Splunk is on track to become a billion-dollar company and you, the intrepid sysadmin/developer, are going to help us get there. Now, this is not a statement that I’m making as an analyst who “covers” the enterprise software market, and compiles a list of “top software companies to watch”. I’m writing this as Splunk’s Platform Architect, a techie whose goals are to ensure that what comes out of our development group is compelling and exciting to those that are actually working with the product.

It is this developer-centric ethos that sets us apart from so many of the other enterprise software firms and has already paid dividends on community goodwill. Instead of making prospective buyers jump through registration hoops just to view a guided webcast tour, Splunk provides fully functional software downloads to try out on your own data, inside your own network, free from webinar smoke and mirrors.

We don’t just want you to try out the software, we want you to try doing things that aren’t covered in our brochureware, things that sound ludicrous at first but are doable. In fact, in a perverse way, we hope that you do break our product because it reveals new limitations for us to solve, ultimately leading to a product that lets you do your job the way you want, yet easier and faster.

Adventures on a mini-bike amongst the boxes in engineering:

External view:

Internal view:

For all you hardcore Web 2.0 fanboys, I’m giving a talk at AjaxWorld on “High-Performance AJAX Application Design” down in Santa Clara at the end of September. The official blurb is:

Designing an AJAX application that meets enterprise scalability and performance requirements presents technical challenges that aren’t addressed by traditional AJAX frameworks. This session will highlight the techniques used in Splunk to address handling large amounts of data in the browser, persistent multi-panel state management, interface customization and localization, and interactive DOM-accessible graphics support. By leveraging existing, though less common, techniques such as iframe-style AJAX, in-browser XSLT, and contextual CSS, modern browsers can provide a compelling interface without the need for a thick-client installation.

Come by and say hi.

On a daily basis, I pay homage to the wonder that is Blue Bottle Coffee espresso, which flows freely — some would say excessively — from our kitchen. The benefits to productivity that this fine coffee bestows upon the dev team is enormous, easily eclipsing other contenders such as video games or foosball. Of course, there were some hurdles to get to this point, namely somebody pouring M&Ms into the bean grinder of the super-automatic that was previously in service. The result was a pitiful molten mess of chocolate, beans, plastic, and gears. And, of course, the perpetrator was never discovered. So the only recourse was to beef up the machinery and move to a true commercial setup: a La Spaziale, Mazzer Mini, and freshly delivered Blue Bottle. BB even asked us what hardware we were running, and sent us the most compatible beans. Brilliant.

My name is Johnvey, I create web sites, and I hate computers.

Many a time have I contemplated forming a support group for computer professionals who’ve grown to loathe the very tools that sustain their livelihoods. Now don’t get me wrong, I love the internets in its wholesomely unadulterated form, dancing hamsters/Star Wars Kid/PBJ time included. But there was a point in the last few years when the honeymoon period was over and I saw computers in a different light — a light that was illuminated by a blue screen of death, or more recently, the multilingual translucent grey screen of death.

I imagine that marriage is an accurate analog to my relationship with computers, and that all my computer problems have degraded from “configuration issues” to “petulant machine that won’t stop throwing this retarded error” (disclosure: I am not married, so my only basis for comparison would be vicarious experiences through eavesdropping on the subway). Countless hours have been pissed away trying to decipher what “node no longer exists” or “pseudo-terminal will not be allocated” really means.