With the previous setup, here’s what I want for my app:

A dashboard with a couple pretty pictures and some top N lists
Saved searches for advanced users to explore further
It should work for all my users with whatever indexes they have access to

I’m going to start with the sample_app template available in Manager and add what I want. Then I’ll clean up the sample stuff I don’t need. So the first step is to create a new app in Manager->Apps. Give it a name and an optional label and select “sample_app” as the template. I don’t have any additional files to upload now, so I’ll leave that alone. Save and I’m back to the list of installed apps.

On the filesystem, a bunch of things just happened. The directory MyGreatApp was created, containing a complete app structure and sample files, enough to have a functioning app. These files are all based on simplified XML that hides much of the complexity of the underlying full XML format. This makes it easier to build views, but has limitations. (For more on this see the docs: Simple Dashboards)

Some highlights:

If you are comfortable editing XML, here’s a handy hack to get the list of your default indexes in the “All indexed data” dashboard. It will show whatever the logged-in user has access to.
If you are using the standard dashboards from the Search app, do this:

Go to $SPLUNK_HOME/etc/apps/search/default/data/ui/views
Copy dashboard.xml to $SPLUNK_HOME/etc/apps/search/local/data/ui/views
Change the permissions on the file so you can edit it
Right before the last </view> tag at the end insert this XML:

 <module name="HiddenSearch" layoutPanel="panel_row2_col1_grp4" group="All
indexed data" autoRun="True">
    <param name="search">| eventcount summarize=false index=* -count</param>
    <module name="SimpleResultsHeader">
      <param name="entityName">results</param>
      <param name="headerFormat">Indexes (%(count)s)</param>
      <module name="Paginator">
	<param name="count">20</param>
	<param name="entityName">results</param>
	<param name="maxPages">10</param>
	<module name="LinkList">
          <param name="initialSortDir">desc</param>
          <param name="labelFieldSearch">*</param>
          <param name="valueField">count</param>
          <param name="labelField">index</param>
          <param name="labelFieldTarget">flashtimeline</param>
          <param name="initialSort">count</param>
	</module>
      </module>
    </module>
  </module>

Save the file.
Back in the UI, click the Splunk logo to refresh the search app.

Presto! Now there is a new column showing indexes. If something didn’t work right, just remove the file you created. This file won’t be overwritten on upgrade, so if in the future there is a change to the search app you will still have this version because files in local take precedence.

I’ve been working on some apps for 4.0 and finally I can talk details. Over the next couple posts I’ll walk though creating a simple app using the new UI tools and a little XML. This is all based off the Apache logs on my server, so first a little background on how I’ve configured my 4.0 instance.

I have a typical small server whose primary purpose is to host a dozen or so low traffic websites. One site gets half my hits, three more most of the rest and the stragglers round out the lot attracting bots. Each virtual host has separate access_log and error_log files but all use the same format: access_common.

To take advantage of the new multi-index search in Splunk 4, I’ve set up my instance to use different indexes for various sources. In my case, it’s by person, as I have several groups of sites managed by a particular admin. The indexes are named www_something so as the overall administrator I can search across all of them with “index=www_*” and still not have to touch the other system events I’ve got going into the main index. I have also set up roles so each admin sees only the relevant data (and isn’t confused by the rest.) All the config is explained in the docs, so I won’t go over it right now.