Blogs

I’ve been asked a few times on how best to search for events which may contain many different discrete values for a field. It’s essentially using an OR (disjunctive search) in the search language. For example, you can do this:

sourcetype=my_sourcetype (planet=mars OR planet=earth OR planet=saturn)

This works fine for a finite case where you only have a handful of planets, but what happens if the field’s possible search criteria changes daily and may contain hundreds of possible values that you would like to input for the search? Certainly, using OR terms with over a hundred entries sounds impractical. A solution is to have an external file that contains all the possible values that you would like to use in the disjunctive…

I visited Sydney and Melbourne last week to host our first Splunk Live events in Australia. Its my first visit to Australia and I’m really blown away by the friendliness of the people we’ve met. And the “Australian for Grep” t-shirt finally had a proper home. Attendees at today’s event in Melbourne and Tuesday’s event in Sydney included an impressive list of current customers and partners and a number of new users evaluating Splunk for the first time including Telstra, Ericsson, InfoSys, Frontline Systems, Fujitsu, GE Capital Finance, Toll Holdings, Vanguard Investments and more. We owe a huge thanks to the team from Digital Networks Australia who sponsored the two events.

Martin Brown, A Large Australian Financial Services Company

In Sydney Martin Brown,…

Ok, here’s a real blog post to make up for that last one. You may have heard that one of the major features of Splunk 4.0 is a brand new REST API. This is the interface that both the CLI and the web UI use to manage Splunk inputs, retrieve splunkd status, perform searches, etc. You, too, can use this API for doing all sorts of good or evil – read on.

Explore a bit…

Exploring it is easy – point Firefox at your your local Splunk instance’s management port. For example, https://localhost:8089/services is the default. Adjust https vs http as necessary, as well as the port. Note that this is the management port, not the web interface port (which is 8000 by default).

In a decent…

This will be a very brief post, to fulfill my obligations. I’ll share something a little more informative, perhaps even more interesting, in an upcoming post (soon… I promise (kinda) this time).

As of Splunk 4.0, our old somewhat-of-an-API has been replaced with an entirely new REST API, invalidating my old post on reloading authentication from the command line.

Sooo….. in 4.x, you can restart the authentication system with the following command:

$ splunk _internal call /authentication/providers/services/_reload -auth admin:changeme

Any errors should be obvious in the resulting XML. As of 4.0.3, you’ll also get a non-zero return code upon receiving errors from the API. And I’ve filed a bug (just now) to expose this as a real CLI command, so soon this post will…

Last week we continued our road show launching Splunk 4 through the Southwestern US in Phoenix, San Diego and Los Angeles.This was our second annual gathering of customers, partners and users and we had more than double the attendees at this year’s Splunk Live events. In the morning we held a three-hour hands on technical workshop. Attendees had the opportunity to install and configure Splunk 4 on their laptops or remote server and get one-on-one assistance from the Splunk team. Afternoon sessions and dinner focused on customer presentations. We’re very grateful to all the presenters who took time out of their busy days to share with everyone how Splunk is transforming their IT environments. I captured some notes from the…

Many companies produce RSS (Really Simple Syndication) feeds for their employees, partners, and customers. Moreover, these same companies consume RSS feeds from their suppliers whether it be personal news information or more timely business data. RSS is a great way to digest this information, but after a certain period, it may not be possible to find it again. If information from a RSS feed were indexed on a regular basis, say every 10 minutes to 30 minutes, into Splunk it could be searched at anytime. To accomplish this, I’ve created a simple Splunk application to index some RSS metadata (date, title, link, and description) on Splunkbase. Simply download the application and install it into your $SPLUNK_HOME/etc/apps directory. Then, modify its inputs.conf file.…

Now that I’ve (hopefully) convinced you that ldapsearch is your friend, let’s get down to the matter. How can you use that information to configure Splunk to authenticate against LDAP?

The file used to configure LDAP authentication: authentication.conf

If you have never attempted to configure ldap auth before then you won’t have one of these files in your $SPLUNK_HOME/etc/system/local/. You can either create it by hand or use the UI (which creates the file for you.)

Here’s a sample authentication.conf file that I will break down for you. (BTW, a lot of this explanation already exists in a file called $SPLUNK_HOME/etc/system/README/authentication.conf.spec):

[authentication]
* This does not change
authType = LDAP
*If you want LDAP, set it to LDAP. Other options are Splunk and Scripted.
authSettings = myldapstrategy
*the…

I’ve had this topic come up in several technical conversations lately, so I thought I would blog about it now.

Situation: You have two different source types containing common key field values, but the actual name of the field itself is different within each of the source types.

Question: How do you produce a report within Splunk that correlates all of these fields values together under one normalized field name?

Answer: Use the new FIELDALIAS and EXTRACT features included with Splunk 4.0 to normalize the field name at search-time.

Example: Let’s suppose you have two different types of call detail records, each containing a number that represents the total duration in seconds that someone is on a phone call.

One CDR event looks like this:

TELCOE,2.1,7e197787-655330a9-7a458301-70845177@12.13.20.20,,0,,H,,S,,sip:7622550@127.10.15.17:5050, sip:5558889999@120.10.20.20:55555,TELCO:Dallas,TX,0,sip:7622555@110.130.52.25:5050,NORTH:NORTH,200,0
,1,0,1,0,08/02/2009:05:03:21,08/02/2009:02:03:22,92,UNKNOWN,0,0

and the…

If you’ve ever tried exporting lots of events from Splunk UI then you probably know that there’s a hardcoded max of 10,000 lines. This is to prevent users from potentially crashing splunkd or python. Taking the previous into consideration may allow you to view this restriction as a safety feature.

In most cases, users should not need to export 10,000 lines of data. If you’ve got more than 10,000 lines, you should refine your search so that your have less (a lot less) than that. There are probably a few cases where there’s a legitimate reason to export this many lines and more. If you feel the compelling need to be able to export more than 10,000 lines here are…

With the previous setup, here’s what I want for my app:

A dashboard with a couple pretty pictures and some top N lists
Saved searches for advanced users to explore further
It should work for all my users with whatever indexes they have access to

I’m going to start with the sample_app template available in Manager and add what I want. Then I’ll clean up the sample stuff I don’t need. So the first step is to create a new app in Manager->Apps. Give it a name and an optional label and select “sample_app” as the template. I don’t have any additional files to upload now, so I’ll leave that alone. Save and I’m back to the list of installed apps.

On the filesystem, a…