Blogs

Changes made in /etc/system/local override any configuration bundles that you may be trying to publish to your Splunk instances using a DeploymentServer.

Serveral customers have reported that DeploymentServer configuration bundles were not taking effect, only to realize after several troubleshooting cycles that there was some configuration in /etc/system/local that was preventing that from happening. Note that any configuration in /etc/system/local will always take precedence over any other configuration in the system – even deployed bundles.

So, if you are stuck in this position, please make sure to check your /etc/system/local before hitting the panic button!

Salutation drivers of the Information Super Highway,

I’ve got another post here in the occasional “Help Me Help You” series, this time I’m going to digging into case writing.

I was talking with the some of the engineers the other day around the bar about an issue that one of our field guys opened. One of the engineers mentioned a piece of information that totally changed the way the rest of us were going to handle the issue. This got us to talking about how some people write great cases and others don’t. The ones who write good cases usually get their issues resolved first (often times closing the issue with the first response from a member of my team), the ones…

I’ve been playing with a few things that will eventually turn into an iPhone application to talk to Splunk via the REST API. I don’t have a lot to say about it right now due to other issues but I do have a little something to show off:

Splunk doesn’t support Safari officially yet and MobileSafari is a whole ‘nother animal, but there are other things you can do. You can talk to the REST endpoints just fine. Here I have a Live Tail search running from the browser, talking to my production server.

In 3.2.x and 3.3.x, dashboards refresh automatically on their own schedule: 10% of the time period or 1 hour, whichever is sooner. You can’t change this right now. But if you want to force a refresh, you can delete the files that contain the cached data.

Dashboards create username_* files in $SPLUNK_HOME/var/run/splunk to persist the dashboard data. There is also a directory for each username with *.csv files. Delete the username_* files (like “admin_KB indexed per hour last 24 hours”) and the *.csv files and the next time you refresh the dashboard, it will reload.

This is not an elegant solution by any means, but it does work. While you could just delete the files for the search in question,…

Everyone has their favorite customer.
I have one too and he is the CTO of a very cool IVR/VoIP platform. His name is RJ Auburn

Around here is synonomys with filing 34 bugs between sunday 9PM when we push bits to the site and 9AM when we get in to the office. I dont mean the usual the UI-is-off-by-10-pixels but complex indexing or distributed search bugs. Well, sometimes is its a trivial thing we missed, but usually he is usually pushing splunk to its limits. Its not often that a CTO and “industry expert” is the one to personally put splunk through its paces – but it’s RJ is like that and gets his hands dirty – and splunk is the better…

We here at splunk are into processing lots of data. Our external marketing focuses mostly on hardcore IT data but internally we play with all sorts of data sets : government stats, sports stats, even music as shown by Brian cool post.

I just wanted to congratulate Nathan over at FlowingData for crossing the 3100 subscriber mark.

FlowingData is a fantastic example of the hidden value in the data all around us. As more and more of what we do is documented by computers the impact of statistics has become less of a hard-core math geek sport and more within the reach of anyone’s curiosity. His daily posts are a constant reminder of how statistics has become a crossover genre.

Thank you…

Since so few of you, especially the ones that complained about ghurrly drinks, came and drank gin last week there is still plenty of gin.

So now we’ll try broadening our audience by watering down perfectly good gin. But fear not, PM can complicate even this simple feat.

Gin and Tonic drinkers, especially at a bar, can be sorted with high probability into one of these two groups.

• Really wants a vodka tonic or other taste-free beverage, but is unable or unwilling to ask for it in public.
• Actually likes gin, and is adding tonic in order to drink more of it, faster.

There are subtle variations, but this sorting function works with stunning accuracy.

Now both are fine categories, but…

I feel like I should post a follow-up to my recent post about SIM is dead. Here are some points I would like to clarify:

• If I talk about SIM or SIEM, I am talking about the way current SIM solutions are working and the way they are implemented. That means things like relational database, fixed schema, parsed and normalized data, or hierarchical scaling.
• Do I really believe that SIM is not useful? No. And I am not just saying that because I own stock in a SIM company. Just like Alex says in a comment on my original blog entry: IDS is not dead. SIM is probably not dead either. I know of quite some people that are very happy with their…

What do you do if your system administrator locks you out of your critical systems, changes the root password and then quits? If you haven’t thought about this, you are not the only one. San Francisco officials are facing exactly that question. A disgruntled employee locked out all the system administrators from some fairly critical systems, as you can read in the San Francisco Chronicle.

Insider crime is an area in computer security that still doesn’t get much attention. One of the problems is that the frequency of incidents is fairly low and therefore the problem rates low on a company’s charter. However, the big problem is that the average cost of such an incident is really high. In reality, companies are…

I wrote a WordPress plugin (tested for 2.5.1) that displays my most recent Google search terms in my sidebar. It was an experiment with using the Splunk REST API and the PHP SDK.

You can configure the widget from the Widgets page and it supports multiple instances with different configuration. Right now the actual search string is hardcoded because I’m doing some extra mangling to get the search terms the way I want anyway, but I’ll be adding that to the configuration options also. Eventually there will be a way to cache results so you don’t do the search each time the page is loaded.

Since there is still work to do to make it more generic, I haven’t uploaded…