Pick Up Where You Left Off In Scripted And Modular Inputs
Splunk is really good at knowing what has been read and what is new when dealing with machine data that is on disk (using the fishbucket). However, there is a lot of machine data that does not exist on disk. Some examples of this type of data are 3rd party APIs or in-memory data. In order to get to this data, we often use scripted or modular inputs.
When dealing with scripted or modular inputs, it is important to only get new events or information since the last time the input ran. You don’t want to keep indexing the same events over and over because this will cause index bloat and performance issues. So, you need a way of …
Splunk, Big Data and Healthcare Analytics in the Federal Government – Part 3 DHMSM
Welcome to part three of my three-part blog on the ascending role of big data for healthcare analytics in the federal government. In this final part of the series we look at DHMSM, a very large project to find efficiency and insight in near real time. Part one and part two can be found here and here.
DHMSM and the problem to be addressed
Department of Defense Healthcare Management System Modernization (DHMSM) Program is administering an RFP for a potential $11B effort which calls for the modernization the Department of Defense healthcare system by uniting multiple legacy healthcare systems and data stores, developed over decades. I’ve reviewed most of the RFP consisting over 20 attachments which also calls for …
Trade Me: Using Splunk for Multi-channel Analytics
From one side of the globe to the other, I continue to be impressed with the innovation and success of Splunk customers. In my last post, we discussed the success of UK retailer John Lewis and today we’re headed over to New Zealand to check out the innovation of online marketplace and classified ad site Trade Me.
Like many Splunk customers, Trade Me initially downloaded Splunk to address IT operations challenges (covered in detail in our Trade Me case study). As the company found success with Splunk, the Trade Me team realized that correlating clickstream data with structured data could uncover new business insights.
By mashing up clickstream and mobile data with structured data from relational databases, Trade Me …
Identifying Zombie, Chatty and Orphan VMs using Splunk App for VMware
Virtualization is difficult to manage given the complex moving parts from storage to networking to hardware. When you have a dynamic VMware environment with Distributed Resource Scheduler (DRS) and High Availability (HA) enabled, Virtual Machine’s (VM) in the environment can transition through multiple hosts and clusters and can potentially become unregistered VM’s. This can lead a VMWare Administrator to loose visibility for these VMs. In addition each VM in a datacenter could cost from a couple hundred dollars into the thousands (http://roitco.vmware.com) based on your environment and infrastructure costs.
In this blog post I will cover three types of VM’s that can exist in your VMware Infrastructure and requires additional attention. The definition of these VM’s vary, but I’m sure …
I’m somewhat of a Heroku fan boy. I’ve been using it for some time because it is just so simple to deploy applications. However, I’ve never really looked too deeply into the logs produced by my apps via the command line.
In this post we’ll look at how you can start Splunking data from apps deployed in Heroku, and some recipes to visualise it using the SPL.…
Splunk Answers is now migrated!
Splunk Answers has just been migrated to a new platform! Read more about the process and goals.
What to expect
You won’t see much in the way of UI changes, but the site underneath will be more stable and more flexible. You should experience faster loading times, more responsive controls, and very importantly, an improved search experience. We will now also have access to new and improved spam blocking features, a much-needed improvement.
The goal of the initial migration is to maintain feature parity with the existing Splunk Answers site. This will help us make sure we don’t break anything you’ve come to rely on. Over time, we will be able to launch new features and improved functionality.
Monitor and reclaim valuable disk space on Microsoft Exchange Server
While disk spindles get cheaper, disk space on servers hosting mainstream services like Email or Messaging Service, still remains a big budget item. As organizations continue to grow and more people join hands (employees, contractors, service providers, developers, et al), it is important for organizations to monitor and make optimal usage of the critical disk space.
In the Infrastructure and IT Operations space, Microsoft Exchange continues to retain top-spot in the Gartner’s Magic Quadrant for Unified Communications report. Splunk App for Microsoft Exchange provides valuable insight regarding various aspect of Microsoft Exchange deployment landscape.
Splunk App for MS Exchange provides granular insight regarding the complete lifecycle of an email right from the time an email arrives within an org …
Big data and the business of higher education
There was a nice article published on GovDataDownload today about the potential for big data to impact the business of higher education. The material does a nice job of explaining big data in simple concepts, then cites an excellent example of how it can help the bottom line of a university directly. Perhaps more importantly, the article closes with a mention of big data being used to help with learning analytics by “helping identify predictors and patterns for student success”, which is near and dear to my heart as a former educator.…
Using Flume to Sink Data to Splunk
If you have ever used Splunk, you can probably come up with a number of reasons why you should use a Splunk forwarder whenever possible to send data to Splunk. To quickly illustrate some of the benefits, a Splunk forwarder maintains an internal index of where it left off when sending data. If for some reason the Splunk Indexer has to be taken offline, the forwarder can resume its task after the indexer is brought back up. Additionally, a forwarder can automatically load balance traffic between multiple Splunk indexers. There’s already a Splunk blog here devoted to getting data into Splunk that highlights a forwarder’s benefits that I encourage you to review.
But what if using a Splunk Forwarder is …
New Splunk Tools for .NET Developers
Today we’re releasing a new suite of tools for .NET developers so you can supercharge your .NET development with Splunkl!!
CC image Supercharger by Eaday on Flickr
This release is a continuation of our commitment to provide developers a rich platform for developing Splunk solutions.
- C# SDK 2.0 – A new, modern, C# SDK for building cross-platform solutions that consume Splunk’s API and/or which extend Splunk.
- Logging libraries – These libraries allow you to easily wire logging in your existing .NET applications to send log data to Splunk via TCP or UDP. It provides .NET Trace Listeners as well as sinks for the Semantic Logging Application Block (SLAB).
- Visual Studio Extension – This extension makes it really easy to get