If you are comfortable editing XML, here’s a handy hack to get the list of your default indexes in the “All indexed data” dashboard. It will show whatever the logged-in user has access to.
If you are using the standard dashboards from the Search app, do this:

Go to $SPLUNK_HOME/etc/apps/search/default/data/ui/views
Copy dashboard.xml to $SPLUNK_HOME/etc/apps/search/local/data/ui/views
Change the permissions on the file so you can edit it
Right before the last </view> tag at the end insert this XML:

 <module name="HiddenSearch" layoutPanel="panel_row2_col1_grp4" group="All
indexed data" autoRun="True">
    <param name="search">| eventcount summarize=false index=* -count</param>
    <module name="SimpleResultsHeader">
      <param name="entityName">results</param>
      <param name="headerFormat">Indexes (%(count)s)</param>
      <module name="Paginator">
	<param name="count">20</param>
	<param name="entityName">results</param>
	<param name="maxPages">10</param>
	<module name="LinkList">
          <param name="initialSortDir">desc</param>
          <param name="labelFieldSearch">*</param>
          <param name="valueField">count</param>
          <param name="labelField">index</param>
          <param name="labelFieldTarget">flashtimeline</param>
          <param name="initialSort">count</param>
	</module>
      </module>
    </module>
  </module>

Save the file.
Back in the UI, click the Splunk logo to refresh the search app.

Presto! Now there is a new column showing indexes. If something didn’t work right, just remove the file you created. This file won’t be overwritten on upgrade, so if in the future there is a change to the search app you will still have this version because files in local take precedence.

For debugging files getting re-indexed, sometimes what I want to see can only be found in the fishbucket index of the affected instance. I can pick up and move an entire index (3.x+) and drop it into another instance, but when working with the fishbucket there are a couple other things to watch out for. I don’t want anything to change it once I put it in the new instance. So I set up a throwaway instance to easily make changes I wouldn’t want to do to a real one.

REALLY BIG WARNING

Don’t do this to any Splunk instance you like. You will be unhappy later. Throw away your dummy instance when you are done so you don’t confuse anybody.

Set up a new instance of an appropriate version, the same or more recent as the original and appropriate architecture (ppc/sparc or intel.) Get it all working with the correct ports so you don’t conflict with anything else that may be running on the machine. Since it won’t be indexing, the license doesn’t matter. Start and then stop so the first run stuff is done.

In 3.2.x and 3.3.x, dashboards refresh automatically on their own schedule: 10% of the time period or 1 hour, whichever is sooner. You can’t change this right now. But if you want to force a refresh, you can delete the files that contain the cached data.

Dashboards create username_* files in $SPLUNK_HOME/var/run/splunk to persist the dashboard data. There is also a directory for each username with *.csv files. Delete the username_* files (like “admin_KB indexed per hour last 24 hours”) and the *.csv files and the next time you refresh the dashboard, it will reload.

This is not an elegant solution by any means, but it does work. While you could just delete the files for the search in question, there is no simple way to identify which csv file is associated with it. Just don’t go messing with the other files in this directory, you will be Very Unhappy if you do.

I wrote a WordPress plugin (tested for 2.5.1) that displays my most recent Google search terms in my sidebar. It was an experiment with using the Splunk REST API and the PHP SDK.

You can configure the widget from the Widgets page and it supports multiple instances with different configuration. Right now the actual search string is hardcoded because I’m doing some extra mangling to get the search terms the way I want anyway, but I’ll be adding that to the configuration options also. Eventually there will be a way to cache results so you don’t do the search each time the page is loaded.

Since there is still work to do to make it more generic, I haven’t uploaded it to the WordPress site. But here is the basic PHP code to play around with. In fine programming tradition, I learned quite a lot by picking apart existing WordPress widgets, in this case Random Image and Twitter Tools. This widget requires the Splunk PHP SDK, by default my code is expecting it to be in the same directory (which is probably going to be something like wp/wp-content/plugins/widgetname.) There are a few things it depends on, you can find the details at the Google Code page.

Preview is out the door, woohoo! So up here in support I’m busy with the existing versions so I hadn’t checked out many of the new features. I wanted to mess with real data I care about, so I figured I’d copy my existing index and drop it into my splunkpreview directory. I host a handful of domains at home (on Leopard Server) and I’m using Splunk to watch various things I want to know, like who’s commenting on my blog and how many dictionary attacks I’ve had today. I thought it would be nifty to look at the same data in both 3.1.3 (my current production version) and preview.

The first time I tried it, I thought I’d be clever and set it all up before first startup with my whole index, users, saved searches and basically everything. Because, well, I clone this stuff all the time between 3.1.x versions when I’m setting up repro environments for customer issues. Wrong! Not sure what I forgot, but for my efforts I got a nice big segfault. Well, nothing a little rm won’t fix.