Blogs

This morning I got a nice little LCD from Crystalfontz that allows me to connect to it via the open source project lcdproc. After a bit of compiling and installing, LCDproc (which runs natively on linux, darwin (osx) and most other unix distros) connects to any serial, parallel or USB LCD device. In this case, the Crystalfontz LCD is 4 line by 20 character display.

Once configured and connected, you start the server and accept connections.

I then grabbed the IO-LCDproc perl module and modified it to display to the LCDproc server. You can get the IO-LCDproc through CPAN.

Here is the code that would go in your $SPLUNK_HOME/bin/scripts directory

[source:python]
#!/use/bin/perl -w
use IO::LCDproc;
use IO::Socket;
use strict;

&usage if (! $ARGV[0]);

my $client = IO::LCDproc::Client->new(host => “localhost”, name =>…

There has been a lot of talk about the Splunk Platform of late, but what exactly does it mean when we say we have a platform? I figured this would be an interesting question to spring upon unsuspecting members of the development team, and here’s what they (and I) had for our answers:

[quicktime]http://blogs.splunk.com/devuploads/2008/02/splunk_as_platform_large.mov[/quicktime]

Browsing over on Wikipedia, one excerpt states that “a platform describes some sort of hardware architecture or software framework”, and the description for a software framework, says it “may include support programs, code libraries, a scripting language, or other software to help develop and glue together the different components of a software project”.

A platform can be considered as a type of framework – one which helps developers write software…

If you’ve read my previous post on delimiter based KV extraction, you might be wandering whether you could do more with it (Anonymous Coward did). Well, yes you can, I am going to cover the “advanced” cases here. Before covering the capabilities, as in other posts, I would first go over some observations and examples.

Observations

1. Header-body. Some applications, for different reasons, choose to format their log files using a header and a body section. The header usually describes the way the fields are organized in each logged event, while the body consists of logged events, usually one per line, with field values delimited as described in the header. W3C, CSV etc come to mind, see examples
2. Single-delimiter. Other applications choose to use…

As described in my previous post, key-value pair extraction (or more generally structure extraction) is a crucial first step to further data analysis. While automatic extraction is highly desirable, we believe empowering our users with tools to apply their domain knowledge is equally important. To this end, this post introduces one of the simplest forms of key-value pair extractions (KV-extraction) – delimiter based extraction.

Observation

Most logged events usually contain a list of key-value pairs (e.g. attribute list, method call values etc) in a context-dependent well-defined format. An example of well-defined format: ” key-value pairs are separated from each other using ‘;’ while the key is separated from the value using ‘=’ “. More generally, well defined attribute listing formats are…

The common event expression (CEE) effort is moving along. If you haven’t seen much coming out of CEE, it is not that we are not working on it. We have been busy defining and hashing out various aspects of the CEE standard. I am getting ready to release a list of fields for the syntax part of CEE. The taxonomy is moving along as well and I am compiling the final pieces to release for discussion.

If you are interested in the public discussions around CEE, the Mailing list archives are now online.

By Raffael Marty

When you come to dev.splunk.com, you see pictures of beer pong, full bars, stuffed ponies with fart machines taped to their ass, etc – basically engineers gone wild. Somewhere between all of this insaneness, we actually find the time to write code and solve problems like this one.This post is all about a crazy-weird performance issue that we were experiencing, how it manifested itself and ultimately how it was fixed.
I suspect others may be having this problem, as the problem lives in some very popular open source code as far as I can tell. With that, I’ll begin telling you about my journey into hell.

Splunk has a home grown embedded HTTP(S) server that serves up all external interfaces to the ’splunkd’…

Recently, Johnvey Hwang wrote a post called Standing on Our Own Platform. He was the first one at Splunk to break the ice and use the “P” word. Now it’s out there. What do we see when we stand on our own platform? While only you and the future will tell us — there are a few things we hope to see on the horizon.

First, it’s our belief there’s a lot of money out there wasted on point products for managing networks, servers, applications … even security. A lot of these systems redundantly collect, transmit and store much of the same machine generated data. Think of the network, storage and administration resources duplicated on all this stuff. By providing…