I pondered at times when I was testing early versions of the UI changes that Splunk was courting a different demographic and losing touch with sysadmins. The same sysadmin/networkadmin user base that put Splunk on the map.

My concerns were myopic at best.

In testing the command line interface, getting to know REST, installing Splunk hundreds of times in various configurations and seeing the work that is being done with the various SDKs I understand Splunk as a powerful tool that empowers users in more areas of a business than ever.  Especially those same systems and network oriented people that saw the most benefit early on.  While some of the new UI features and workflows aren’t something I thought I needed back at my last employer, they sure would have helped the executive staff see benefits more tangible to their scope.  Benefits that get around to helping everyone; from systemic visibility at the executive layer, to investigative power at the administrative layer, to improved efficiency at the bottom line.

For those not in the know, the Splunk App for VMware has been a popular request from nearly all of our customers and a much-searched for app. Why? Because virtualization, as much as it allows resources to be shared more efficiently, also leads to problems being shared! Splunk customers want to be able to tie user level or application level problems with the underlying server, storage, network or virtualization layers. They want a way to make those connections easily and at scale.

There is also another big reason to want virtualization layer data in Splunk. Virtualized environments are dynamic. Virtual machines move from host to host or even from one storage location to another.  When someone reports a problem, its not enough to look at the current state of your environment – you really want to go back in time to when the problem first started. You want to see where the virtual machine was, which other virtual machines were on the same host or storage location, was there a “noisy neighbor”, what the host’s performance metrics looked like, what changes  were subsequently made to the host or virtual machine over time and more.

Including virtualization layer data alongside all other data from your applications, your storage, your networks in Splunk,  allows you to really perform correlations across different layers of your technology stack. You can use virtualization data to not only diagnose and resolve performance or operational problems ,  you can also use it to generate operational analytics such as capacity utilization, trending, usage reporting and planning. When data is collected comprehensively, it also becomes relevant for audit, compliance and security monitoring and reporting.

The Splunk App for VMware simplifies the collection of virtualization layer data and getting it into Splunk. And it does this without interfering with the operation of vCenter Server (often the bottleneck) in VMware environments. The solution includes a virtual appliance that collects metrics & logs directly from your ESX/ESXi hosts, as well as topology, tasks, events and log information from VC. Since we don’t collect performance metrics from VC, we can go deeper and collect metrics from the host ESX/ESXi servers at a much lower level of granularity (20 second granularity).

Splunk not only collects ALL the data, it also stores it (unfiltered or summarized) for as long as you need it. You can run all kinds of analyses and correlate data from your virtualization layer with data from your virtualized applications, as well as from your physical infrastructure such as storage and networks. As one of our customers says it, “You never know what data you will need till you need it”.

If you’re familiar with Splunk, you can take some of our example views and reports and generate any dashboards you like for your environment. Some examples we provide are:  total disk used by snapshots, virtual machines with too many snapshots in the environment, thin provisioned datastores with too little capacity available – the list goes on and on.

I am posting a few screenshots here and will plan to do a webcast and demo shortly. Stay tuned for more good stuff, and as usual, if you have comments or questions – email me!

Track statistics as virtual machines migrate

Detailed performance metrics, for hosts and virtual machines (disk metrics in the below picture)

Capacity Reporting

Detailed Log Analysis And Reporting

Our next meeting will be held at the Splunk Office in Plano, Texas on Tuesday, February 21 @ 6:00p CST.

If you are interested in attending now, please click this link below for details:

http://www.meetup.com/Splunk/Plano-TX

I believe Rick Curry from JCPenney will be presenting something interesting and Splunky during this meeting, so make sure to come.

At least for the beer and pizza!

Besides, I look forward to hearing about your various war stories regarding Splunk. How you work through issues, figure things out, extend/expand your use and, more importantly, your thinking about Splunk. It’s quite an eye-opening experience for a veteran Splunker like myself to learn from you guys and I’m never short of amazed at the creativity that you demonstrate as you leverage Splunk for all kinds of IT problems, apply advanced analytics and correlations now in ways that are actually helpful for a change.

So, needless to say, I’m happy that you want to get together now on a regular basis and I can’t wait until 2/21/2012. See you there!

BTW, I created a Dallas Splunk Users Group Home and Notes page, which can be found here:

Splunk Dallas Users Group Home
Splunk Dallas Users Group Meeting Notes

I also created a Google Group as well, which can be found here:

Dallas Splunkers Google Group

Sign up and come join us, if you want (dare)!

Happy shopping!

Turning off Urchin will be a setback to some users that are looking to do web analytics behind the firewall.  There are needs from Government agencies or other organizations who need to stitch/correlate data with other data sources or want to understand bot/scrapper activity that is easily available in the log files.

We think Splunk might be able to help Urchin users. We have built the Web Intelligence App, which I firmly believe can achieve and exceed the value of Urchin. The Web Intelligence App from Splunk provides all the classic web analytics measurement (traffic, pageviews, content usage, content engagement etc.) with the ability to correlate clickstream data with other data sources (offline, telecom, social, mobile) as well as IT operations data. Real-time reporting of web activities is available in addition to deep data drilldown at a user or session level.  Unlimited segmentation using the powerful Splunk search language (we have a search construct like a search engine) helps an analyst dive deeper into the data. The Splunk Web Intelligence App provides Web Analytics + Operational Intelligence … delivering real-time insights into what’s really happening within the online channel.

Who is using the Splunk for Web Analytics?  Well, we have over 1000 downloads for the app. Some customers are using our core product and some use the App.  Customers like Expedia, Zulily,  NPR are just a few of the exhaustive list that benefit from the use of Splunk for Web Analytics.

I can keep writing another few pages, but would prefer you check out the App which is a free download on Splunkbase.  Did I mention that Splunk itself is a free download? Enjoy!

This recognition is an important milestone for APM overall and for Splunk. In the past, Gartner has categorized it into 5 sub-segments – End User Experience Monitoring, Runtime Application Discovery, Modeling and Display, User-defined transaction profiling, Application deep-dive component monitoring and Analytics. However, as customers need increasingly to understand end-user experience from end to end in complex application environments, their requirements from APM progressively require the ability to mine application data for analytics around customer behavior, customer experience, operational capacity planning and more.

A majority of Splunk customers use Splunk in the context of monitoring their large scale, distributed mission critical applications. They use Splunk in many different ways: to isolate problems, diagnose and troubleshoot issues, to monitor performance and service levels, to connect transactions across different components of their infrastructure and to provide operational insights about their application that aids in IT and business decision-making. Many of these uses would traditionally have been categorized as “Application Performance Monitoring” except that Splunk does more than just monitor the application – it also makes the data relevant to operational decision –making.

Take, for instance, one of our customers – a leading provider in healthcare management systems, who had tremendous challenges parsing their event data to gain a real-time perspective on the current status of their applications and infrastructure.

Even with their initial free download of Splunk, they found they were gaining far more insight into their operational status than some of their existing monitoring tools. They used Splunk to implement performance monitoring with percentage based thresholds on their applications and infrastructure. Every time the response time started to spike, the issues were addressed proactively, before it impacted their customers. Not only did this facilitate breaking down silos in the organization by providing visibility to development teams on areas of the application that were causing these performance degradations, it helped them avoid severe failures and costly infractions on their SLAs.

Shortly thereafter, Splunk became a standard part of the application/infrastructure roll-out process for this customer along with production management and monitoring. With extensive dashboards providing role-based access to various users (such as developers, operations staff, CIO…), they are now able to provide relevant real-time access to information across their enterprise. Several operational decisions like planning for additional capacity during cyclical highs are driven by Splunk reports.

This type of customer experience is pretty typical with Splunk. See some of our previous blogs http://blogs.splunk.com/2011/09/28/the-splunk-revolution-comes-to-europe/  and http://blogs.splunk.com/2011/06/28/buckeye-state-blogging-splunklive-columbus/

Splunk utilizes the data exhaust of applications (logs, events, metrics) and makes it useful in a myriad of ways.

To give you a taste of how Splunk fits into the evolving definition of Application Performance Monitoring, listed below are some snapshots from one of our customers, who is the world’s largest ticket marketplace, on how they are using Splunk for Application Performance Monitoring, in real-time.

Going back to the APM Innovators report, which is why I started typing this rather lengthy blog post in the first place, this is great news for Splunk. Customers looking to enhance their application monitoring solutions can not only point to the success stories from existing Splunk customers, but also a third-party recognition from a renowned research and advisory organization like Gartner. With the definition of APM further evolving in the next few years, the recognition is welcome & gratifying!

Gartner, Inc., APM Innovators: Driving APM Technology and Delivery Evolution, W. Cappelli, J. Kowall, December 22, 2011.

Our aim at SplunkNews is to keep things as centralized as possible. Splunk has a wealth of content that we think you’ll find interesting. On a regular basis, we will bring you up date you on the latest news from Splunk and interesting vignettes that might not make it into a press release but might be worth talking about.

One item that is still very top of mind here at Splunk is last week’s rollout of Splunk Enterprise 4.3, our flagship software for real-time operational intelligence.

You can find the press release here.

For the more visually inclined, check out this video:

Splunk 4.3 Overview

And if you haven’t figured it out yet, we love talking about our products. Check out these four blogs about 4.3:

• Customers speak out on 4.3
• Faster and Insightful Web Analysis
• Splunk even more data with 4.3
• Three features  security pros should start using today with 4.3

Finally, to keep tabs on what we’re up to, check out our Twitter feed: @SplunkNews

Splunkbase now has 229 apps and add-ons (and growing!), and it’s time for a better way for our users to see the quality of a given app at a glance. The solution? Rate your favorite apps by assigning them a star rating.

You don’t have to write a text review of an app to rate it (although we’d really appreciate your honest feedback). Just hover your mouse over the number of stars you want to give an app and click.  That’s it!

If you decide to also post a review, you will see your star rating next to the review.  If you change your mind later, you can always update your review and rating of an app.

If you would like to see the overall rating for an app, it’s on the right, next to the total number of reviews.

Star rating is an easy and effective way  to provide feedback to app authors and to let other Splunkbase users know about the quality and usefulness of an app.  I hope you will find it helpful while exploring our apps.

As always, I greatly appreciate your feedback. Please don’t hesitate to send me your suggestions by email.

Fellow Splunkers,

Yes, the old proverb is still true – there is perhaps nothing that gets the heart racing quite like… announcing new security features in enterprise software!  So fasten your seatbelt while I tell you about some of the exciting new features that made it in to Splunk 4.3.

All of these changes pertain to Splunk Web, which is the application server that you visit every time you point your browser at your friendly neighborhood search head, usually on port 8000.

Configurable Cipher Lists!

One of the biggest complaints that we get from customers usually stems from a ding received during a vulnerability scan or penetration test.

In these cases, customers report that Splunk Web supports weak ciphers, and ask how they can specify a valid cipher list such as they were able to do for splunkd via the cipherSuite setting in server.conf.

With Splunk 4.3, it is now possible to specify the list of ciphers that should be allowed in web.conf via the the cipherSuite parameter:

cipherSuite = <cipher suite string>
   * If set, uses the specified cipher string for the HTTP server.
   * If not set, uses the default cipher string
     provided by OpenSSL.  This is used to ensure that the server does not
     accept connections using weak encryption protocols.

For example, to set Splunk Web to only use TLS version 1.0 cipher suites, set the following in web.conf and restart Splunk:

[settings]
cipherSuite = TLSv1

Non-persistent Cookies!

Another common complaint from customers was that Splunk Web cookies were persistent.  In other words, the cookies were set with a future expiration date, which meant that they would often persist even after the browser was closed.

This was a problem for some of us paranoid folks, as it meant that the Splunk Web session key was persisted on disk beyond the life of the browser session.  Thus begat tools.sessions.restart_persist in web.conf:

tools.sessions.restart_persist = [True | False]
    * If set to False then the session cookie will be deleted from the browser
      when the browser quits
    * Defaults to True - Sessions persist across browser restarts
      (assuming the tools.sessions.timeout limit hasn't been reached)

For example, to set Splunk Web not to use persistent cookies, set the following in web.conf and restart Splunk:

[settings]
tools.sessions.restart_persist = True

HttpOnly and Secure Cookie Flags!

Finally, we heard a lot from folks who wondered why we didn’t offer the ability to set two simple cookie flags in order to help mitigate risk from attacks on a few common vectors.  These were the HttpOnly and Secure cookie flags, which are both now configurable via web.conf:

tools.sessions.httponly = [True | False]
    * If set to True then the session cookie will be made unavailable
      to running javascript scripts, increasing session security
    * Defaults to True

tools.sessions.secure = [True | False]
    * If set to True and Splunkweb is configured to server requests using HTTPS
      (see the enableSplunkWebSSL setting) then the browser will only transmit
      the session cookie over HTTPS connections, increasing session security
      * Defaults to True

For these new settings, we have enabled them by default, so there shouldn’t be anything else you need to do other than to upgrade to 4.3.

/End Excitement

Take a deep breath and try to get your heart rate down.

Happy Splunking!  Feel free to drop us a line via support or answers if you have any additional features ideas or questions.

We grouped the features around three main focus areas: 1) making Splunk easier to use, 2) making Splunk faster and more scalable, and 3) making Splunk easier to administer.

In this post I want to dive in a bit and tell you how we’ve made Splunk easier to use. Our mission is to make machine data accessible, usable and valuable to everyone. Making Splunk easier helps drive the value of machine data to new IT and business users. It’s also compelling from a ‘big data’ perspective – machine data is, after all, one of the fastest growing segments of big data!

So what have we done in this new release to help our users? For one thing, timelines and charts in 4.3 are now Flash-free. This feature alone received an enthusiastic round of applause at our users’ conference in August where we previewed 4.3. Eddie Satterly, Sr. Director of Infrastructure Architecture and Emerging Technologies at Expedia said: “We have 2,700 users of Splunk and being able to provide dashboards on iPads means we can get more data to more people when they want it.”

John Cervelli, Sr. Director of Product Management at Splunk, demoing Splunk 4.3’s non-Flash user interface on a tablet at our users’ conference

A Systems Engineer from a Global Top 5 Financial Services firm told me that he already uses Splunk to quickly turn around ad hoc requests. What used to take his team 6 months now takes them a day to turn around. He loved the new non-Flash UI 4.3 and said their “mobile workers will be delighted.”

The bottom line for us is about getting visibility and insights to the people that need it – anytime, anywhere.

4.3 Visual Dashboard Editor

We also integrated new charting controls and drag and drop dashboard editing into 4.3, so that users can create and edit dashboards on the fly without coding in XML. Derek Mock, Director of Software Development, Ceryx commented that these easier dashboards, “empowers business users by making them self-service.” This was a common theme in the feedback. Mika Borner, Head of Internet Messaging, Swisscom, stated that he was, “impressed by how much faster it was to change charting views.”

Sparklines Visualizations in Splunk 4.3

Next, we introduced a new visualization called Sparklines, which provide a great way to convey at-a-glance trending of Big Data at a granular level. We got some great feedback from an IT architect at a top 5 nationwide home improvement retailer. He told us: “Sparklines means we can now very quickly spot trends. With a large number of stores nationwide, we have a lot going on. We need to know very quickly when something is going to happen and trending at the event level helps.”

Integrated Real-time and Historical Timeline

One of Splunk’s greatest strengths has been the fact you can integrate real-time and historical data in one dashboard. We’ve taken this a step further in Splunk 4.3 by integrating real-time and historical search results in the same chart. A senior product engineer at a global top 5 media, entertainment and communications company noted that, “Having the context of what happened 30 minutes ago with live data to watch conditions as they continue to happen is very useful.”

Last, but not least, is Per Result Alerting. The same user from the top 5 nationwide home improvement retailer commented on how this feature provides, “flexibility in adjusting the granularity of events triggering help desk tickets.” Mika from Swisscom liked how this feature will help them in monitoring and alerting for service abusers.

As you might expect, we’re just scratching the surface here. We haven’t even talked about improved speed and scale and features that make Splunk 4.3 easier to use and administer. Stay tuned!

Go to our What’s New in 4.3 page to learn more about the release.

Splunk 4.3 is available now. Download this latest version and discover how these improvements make your job easier. And be sure to let us know how it goes!

Happy Splunking!