http://www.macnica.net/lanch/lanch10sm/ca01.html/

*Applause*

This is not your hipster’s app contest – this is a contest about Getting. Stuff. Done.

*Applause*

This is a contest about taking all the cool stuff you already do with Splunk, and showing it off for the world to see! On Splunkbase!

*Applause*

This is a contest about rewarding those who create the coolest, most useful apps on Splunk – and everyone’s a winner!

*Applause*

So come one, come all, package your field extractions, views, dashboards, scripted inputs, and other Splunk mods into apps or add-ons for Splunkbaaaaaaaase! Contest begins on August 1 – enter as often as you wish!

First, when setting up retention/archival policy, it’s helpful to understand the basics of how Splunk manages data. Then below is a visual to help match the exact configuration parameter you need to adjust for controlling the flow of data from one stage to the next.

For most environments, the parameters you will likely need to adjust are few:

• maxTotalDataSizeMB – to apply a size-based retention policy
• frozenTimePeriodInSecs – to apply a time-based retention policy
• maxWarmDBCount – to split the hot/warm and cold directories among separate partitions

Currently, it is possible to set maxTotalDataSizeMB (size-based policy) and maxDataSize (bucket size) via SplunkWeb under Manager > Indexes > MyIndex. All other parameters are set using indexes.conf. Here is a simple example:

[myLovelyIndex]
# set directory location of hot/warm, cold and thawed
homePath = $SPLUNK_DB/lovelydb/db
coldPath = /path/to/another/partition/lovelydb/colddb
thawedPath = /my/computer/lovelydb/thaweddb
# set time-based policy of 1 year
frozenTimePeriodInSecs = 31556926
# set size-based policy of 1 TB
maxTotalDataSizeMB = 1048576
# do not delete cold dbs, instead move them to slower storage
coldToFrozenScript = moveToSlow.sh

For more advanced configurations and backup purposes, the full set of parameters will empower you to take advantage of Splunk’s flexible data management.

I already have lots of examples of great reports, dashboards and the like that various Splunk users built and had a big impact – many from the hundreds of SplunkLive user presentations over the last few years. But I would love to feature any others that any of you would like to share. They don’t have to have literally gotten you promoted – but anything you produced in Splunk that wowed your colleagues are welcome. Screenshots are a huge plus but I’d also love descriptions of what the report or dashboard showed and how it helped your organization. Full attribution promised!

Leave a comment or send me an email at cfrln at splunk dot com.

And I hope to see you at the conference.

Beginning in Splunk 4.1.4, a new option, ‘allowRemoteLogin’, has been added to server.conf to better control access to Splunk’s management port (TCP port 8089 by default).

• For instances of Splunk Enterprise, this option will be set to ‘requireSetPassword’ by default, which will not allow the ‘admin’ user to remotely authenticate if the password is ‘changeme’
• For instances of Splunk Free, this option will be set to ‘false’, which will not allow any remote access to Splunk’s management port.
• Changing the value to ‘always’ will allow remote logins.
  • For instances of Splunk Enterprise, remote authentication will be required.  As such, we strongly recommend changing the default ‘admin’ password
  • For instances of Splunk Free, remote authentication is not available.  As such, we strongly recommending against changing this value on Splunk Free instances.

Example of a remote authentication attempt in 4.1.4:

./splunk search “foo” -uri https://some_server:8089 –auth admin:changeme

Remote login disabled because you have not changed the ‘admin’ password yet. Either set the password, or override by changing the allowRemoteLogin setting in your server.conf file.

This change was implemented to better protect instances of Splunk Enterprise using default credentials as well as instances of Splunk Free that have been intentionally or unintentionally deployed on production servers.

As always, please let us know if you have any questions or comments on this feature, as well as ideas for any other features related to security.

Yours,

The Splunk Software Security Group

Visit Splunk.TV to subscribe or: (in the not too distant future) listen live every friday at 11AM Central Time. To be a part of the show and submit a question email splunktalk@splunk.com. Splunk Talk is hosted by Michael Wilde (Splunk Ninja), Jeffery Blake and Eric “Maverick” Garner.

Visit Splunk.TV to subscribe or: (in the not too distant future) listen live every friday at 11AM Central Time. To be a part of the show and submit a question email splunktalk@splunk.com. Splunk Talk is hosted by Michael Wilde (Splunk Ninja), Jeffery Blake and Eric “Maverick” Garner.

This provides an incredibly easy way to integrate external web sites with events and fields in your data. For instance, if one of the fields in your events happens to be called stock_symbol, as in the name of a financial security, and you want more information about it, you can simply go to Splunk Manager->Fields->Workflow Actions->New and set up a new workflow action. A simple one would be a HTTP GET which resolves the link to http://finance.yahoo.com/q?s=$stock_symbol$. Call the workflow action “Get Stock Price” and now when you have the stock_symbol field on the event viewer within Splunk Web, you can select the drop down and select the “Get Stock Price” option. You can have more than one workflow action per field or event. This opens up the possibilities for getting as much information as you can about an event or field or sending your fields or events to as many places as needed to initiate further actions.

I will outline a few more workflow actions that you can get from Splunkbase that I have created.

RSS Feeds Workflow Actions

Back in 2009 on Splunkbase, I created a simple Python based application that can read any RSS feed and index each entry’s date, title, description, and HTML link. The link field literally has an URL that refers to the article mentioned in the RSS entry. In Splunk Web’s default event viewer, you could see the link field, but you would have to copy and paste it into your browser’s address window to get to it. Along comes workflow actions.

I’ve created three separate workflow actions for the link field of the RSS feed. The first one “Get Article” simply sends the link field to Google to do a search for it. The second one is even easier as it opens up a new window with a HTTP request for the link field itself. This is called with much pizazz, “Get Article From Link.” The third workflow action called “Get Proxy Article” is a little more complex in that it makes a HTTP Post call to a public proxy server, which in turn sends the request to show the original link’s article. You can see what this looks like in Splunk Web below.

Essentially, what this does is turn Splunk Web into an RSS reader as it first indexes RSS events using the RSS application and then allows you get to the article’s contents using the workflow actions. You can get this add-on from Splunkbase.

Whois

Let’s discuss a more IT specific example. In most organizations that use Splunk, an IP address often appears in the events and is extracted as a field. There are quite a few statistics you can get on an IP address such its DNS name, geolocation, traceroute, decimal results, etc. I’ve chosen to call the whois service on the IP address field using a free service at http://www.dnsstuff.com/tools/whois/?ip=$ip$ for the sake of example. If you followed the example above, the explanation for this is the same. You can download this workflow action add-on from Splunkbase as well.

In addition, developer Adam Kahtava, has created a much more comprehensive whois service for testing purposes. As a bonus, the same Splunkbase add-on that I have for the whois workflow action has a dynamic lookup whois Python script that I use to call Adam’s whois service. This will add a new field called, whois, corresponding to the IP address field in your original events. Sample usage is like this:

sourcetype="my_events"| head 10| lookup whoisLookup ip OUTPUT whois


For more information on lookup, please look at past entries in the Splunk blogs.

BPM Workflow Actions

Since this feature is called workflow actions, it makes sense that one of the use cases would be a workflow action that initiates a Business Process Management (BPM) flow. For instance, one of the fields in your events may be a trade ID and the trade was rejected. The workflow action of the trade ID field could initiate a BPM call via HTTP GET or HTTP POST to start a workflow to resolve the issue. The same can be done for retail orders, insurance quotes, trouble tickets, request for repair, etc. The possibilities are only limited by the use cases that your data can support. Below is a simple illustration for the high level architecture for this approach:

Any BPM that calls itself integration-ready will have the ability to communicate with the outside world using protocols such as HTTP. As you can see, Splunk is not only an initiator to the BPM, which is a gateway to a larger Service Oriented Architecture, but it is also the place to index the BPM logs to troubleshoot and provide further workflow actions on subsequent events. Unfortunately, I do not have a Splunkbase example for calling a BPM via a Splunk workflow action as each user will be using their BPM vendor of choice. However, the concepts are the same to initiate this call regardless of the vendor.

I hope this article has given you more reason to use workflow actions and provides a basis for some the possibilities for what you can further do with your data indexed within Splunk.

Visit Splunk.TV to subscribe or: (in the not too distant future) listen live every friday at 11AM Central Time. To be a part of the show and submit a question email splunktalk@splunk.com. Splunk Talk is hosted by Michael Wilde (Splunk Ninja), Jeffery Blake and Eric “Maverick” Garner.

Systex Splunk Solution Center

Just a few hours after arriving in Taipei, I visited the Systex offices and the Splunk Solution center that they’ve built to showcase Splunk’s core technology, Systex’s apps built on Splunk and solutions that they’ve delivered to their customers. This impressive lab hosts dozens and dozens of customer sessions each week. In addition to eight flat panels showing real time dashboards and a wall showing the Splunk advantage, they have a large whiteboard for anyone with a good idea of how or why to use Splunk to share it. This whiteboard was completely full of great thoughts!

SplunkLive Taipei

The Systex team had said that the event would be well attended, but the turnout exceeded my expectations by far. At first, I wondered how all the seats in the huge ballroom would be filled, but just before the event began, the hotel had to bring in row after row of additional chairs. Even after this, there were still plenty of people standing in the back of the room! All in all, about 265 people showed up – a great turnout unto itself, considering that it was 90 degrees and pouring rain! The Systex delegation was ready, willing and able to answer any question that came up for the attendees.

SplunkLive started with Frank Lin, President and CEO of Systex discussing the Splunk and Systex partnership and the breadth and depth of problems that Splunk can solve for customers throughout Asia. This was a great setup for my keynote address which came next.  Frank described a vision of Splunk  “…from data to information, from information to intelligence, and intelligence to income.”

Left: The Splunk and Systex teams (Systex CEO Frank Lin at far right).
Right: 265 attendees listen to the Splunk story!

For the main event, I gave two talks, the first a “bit of Splunk” history courtesy of Erik and Rob, our founders, and then an overview of Splunk and “the cloud” and our upcoming VMware app. Many, many thanks go to Sherry Chao, Splunk product manager at Systex, who performed translation duties. Throughout the week, the themes of private cloud and virtualization kept coming up at customers, so the second talk was very timely. We gave some ideas of problems that both cloud providers and consumers run up against as well as some case studies of customers and how they use Splunk to solve these issues.

Systex announced their new DBcare app that aims to solve the challenges of large-scale database administration. I was given a demo of this at the Splunk Lab in the Systex main offices on the Monday before Splunk live. It’s clear that this technology will help their customers get a good taste of the capabilities of Splunk, anchored in a clear problem with a clean solution. This is just the first of many great apps that Systex is investing in, not just for their own accounts but also for Splunk users in general.

The DBcare daily review dashboard.

After the main event, there was a breakout session for the financial industry. These thought leaders gathered to watch the most concrete and specific case study presentation I’ve ever seen. One by one, our customers went to present the Splunk deployment in their environment, itemizing how data flowed from source devices to the indexes. They also showed the exact searches and reports they were running along with screenshots, pinpointing how Splunk can answer their questions and solve their problems.

Leading FSI customer in Taiwan’s Data Center Operations Dashboard.

And the food…

The way to an engineer’s heart is through his stomach. I just have to say that the meals in Taiwan were absolutely terrific. I haven’t been able to stop raving about the delicious meals that the Splunk and Systex teams treated me to last week.

Many thanks go to our VP Asia Pacific Robert Lau, Matthew Lin and Paul Pang from the Splunk Asia office and Johnny Lin, Sherry Chao, Gandalf Huang, Emy Tu and the rest of the Systex team for a fun and productive trip. I look forward to my next visit.