Have you ever had a situation where you found information on a webpage that you wanted to get into Splunk? I recently did and I wrote a free Splunk app called Website Input that makes it easy for everyone to extract information from web-pages and get it into a Splunk instance.
There are many cases where web-pages include data that would be useful in Splunk but there is no API to get it. In my case, I needed to diagnose some networking problems that I suspected was related to my DSL connection. My modem has lots of details about the state of the connection but only within the web interface. It supports a syslog feed but it doesn’t include …
Splunking Social Media: Tracking Tweets
So you use Twitter and have heard Splunk can do “Big Data”. By tapping into Twitter’s API you can use Splunk to investigate the stream of tweets being generated across the globe.
The great thing about using Splunk to do this is that you have complete control of the data meaning it’s incredibly flexible as to what you can build. A few basic ideas I’ve had include tracking hashtags, following specific influencers, or tracking tweets by location in real-time.
What’s more, it takes a matter of minutes before you can start analysing the wealth of data being generated. This post will show you how.…
Splunking the World Cup 2014: Real Time Match Analysis
As an Englishman I’ve been waiting months – with very high expectations – for the World Cup to come around. Reading fellow Splunker, Matt Davies’ blog post titled, “Splunking World Cup 2014. The winner will be…“, only heightened my excitement.
The tournament is now going into the second week and I’ve been starting to look at the teams, players, and tournament more closely. Which stadium holds the most people? Who’s the top scorer? Which referee hands out the most cards?
With these questions fresh in my mind I opened up Splunk and began to have a look at the huge amounts of information being streamed from the tournament. For this post I’m going to explore real-time match updates; including teams, …
The Splunk SDK for Python gets modular input support
Support for modular inputs in Splunk 5.0 and later enables you to add new types of inputs to Splunk that are treated as native Splunk inputs.
The latest release of the Splunk SDK for Python brings modular input support. The Splunk SDKs for C# (see Developing Modular Inputs in C#) and Java also have this functionality as of version 188.8.131.52 and 1.2, respectively. The Splunk SDK for Python enables you to use Python to create new modular inputs for Splunk.
The Splunk SDK for Python comes …
Making SNMP Simpler
From Wikipedia :
Simple Network Management Protocol (SNMP) is an “Internet-standard protocol for managing devices on IP networks”. Devices that typically support SNMP include routers, switches, servers, workstations, printers, modem racks, and more.
SNMP exposes management data in the form of variables on the managed systems.
The variables accessible via SNMP are organized in hierarchies. These hierarchies, and other metadata (such as type and description of the variable), are described by Management Information Bases (MIBs).
MIBs describe the structure of the management data of a device subsystem; they use a hierarchical namespace containing object identifiers (OID). Each OID identifies a variable that can be read or set via SNMP. MIBs use the notation defined by ASN.1.
SNMP agents can …
Command Modular Input
Simplifying the status quo
I’m often thinking about potential sources of data for Splunk and how to facilitate getting this data into Splunk in the simplest manner possible.
And what better source of data than existing programs on your operating system that already do the heavy lifting for you.
Now this is nothing new to Splunk , we’ve always been able to wrap up a program in a scripted input, execute it, transform the output and pipe it into Splunk.
But rather than going and creating many of these specific program wrappers for Splunk each time you need to capture a program’s output , why not create 1 single Modular Input that can be used as a generic wrapper for …
Getting data from your REST APIs into Splunk
More and more products,services and platforms these days are exposing their data and functionality via RESTful APIs.
REST really has emerged over previous architectural approaches as the defacto standard for building and exposing web APIs to enable third partys to hook into your data and functionality. It is simple , lightweight , platform independent,language interoperable and re-uses HTTP constructs. All good gravy. And of course , Splunk has it’s own REST API also.
The Data Potential
I see a world of data out there available via REST that can be brought into Splunk, correlated and enriched against your existing data, or used for entirely new uses cases that you might conceive of once you see what is available and …
Developing Modular Inputs in C#: Part 2
I’m annoyed at our engineering team, but I’ll get over it. You see, just hours after I posted my first blog post on writing modular inputs in C#, the team up in Seattle released the latest edition of the C# SDK. Within that SDK is a bunch of class libraries that do a much better job than my work on the scaffolding needed to produce a modular input. I highly recommend you go over to their site and dig in to this.
Within this blog post, I’m going to adjust my code to use the new scaffolding and take a look at actually running the code.
Learn More about PowerShell and Modular Inputs
For over five years, I have been working with co-host Jonathan Walz on the PowerScripting Podcast, a weekly Internet radio show. The primary topic of the show is the Windows PowerShell scripting language. We like to talk about news, tips, and resources related to the PowerShell community, but the biggest part of most shows is the interview. We’ve had a wide variety of guests on the show, ranging from prolific scripters who enjoy sharing their work, to PM’s, architects, and engineers from largest software and hardware vendors in the world, including Microsoft, IBM, Intel, NetApp, and more.
Recently, we caught up with Joel Bennett, a Windows PowerShell MVP awardee, who also happens to be my teammate on Splunk’s …
Developing Modular Inputs in C# – Part 1
One of the cool new features of Splunk 5.0 is modular inputs, and we’ve already seen some great examples of this, such as the built-in perfmon gathering modular input and the Splunk Addon for PowerShell. However, the examples that are provided in the documentation are in Python. When I started writing my own modular input, I saw that much of the process of writing a modular input is scaffolding and repeatable. Thus I set out to write an SDK that would alleviate much of the scaffolding and provide a good framework for writing modular inputs. This multi-part series will cover the same process by writing a C# version of the Twitter example from the documentation.