Splunking Social Media: Tracking Tweets

splunk-blog-twitter-dashboard

So you use Twitter and have heard Splunk can do “Big Data”. By tapping into Twitter’s API you can use Splunk to investigate the stream of tweets being generated across the globe.

The great thing about using Splunk to do this is that you have complete control of the data meaning it’s incredibly flexible as to what you can build. A few basic ideas I’ve had include tracking hashtags, following specific influencers, or tracking tweets by location in real-time.

What’s more, it takes a matter of minutes before you can start analysing the wealth of data being generated. This post will show you how.…

» Continue reading

Splunking the World Cup 2014: Real Time Match Analysis

splunk-blog-world-cup-stadium-chart

As an Englishman I’ve been waiting months – with very high expectations – for the World Cup to come around. Reading fellow Splunker, Matt Davies’ blog post titled, “Splunking World Cup 2014. The winner will be…“, only heightened my excitement.

The tournament is now going into the second week and I’ve been starting to look at the teams, players, and tournament more closely. Which stadium holds the most people? Who’s the top scorer? Which referee hands out the most cards?

With these questions fresh in my mind I opened up Splunk and began to have a look at the huge amounts of information being streamed from the tournament. For this post I’m going to explore real-time match updates; including teams, …

» Continue reading

Getting data from your REST APIs into Splunk

Overview

More and more products,services and platforms these days are exposing their data and functionality via RESTful APIs.

REST really has emerged over previous architectural approaches as the defacto standard for building and exposing web APIs to enable third partys to hook into your data and functionality. It is simple , lightweight , platform independent,language interoperable and re-uses HTTP constructs. All good gravy. And of course , Splunk has it’s own REST API also.

The Data Potential

I see a world of data out there available via REST that can be brought into Splunk, correlated and enriched against your existing data, or used for entirely new uses cases that you might conceive of once you see what is available and …

» Continue reading

SPLogger: iPhone Logging API

This week I put up on GitHub an early version of a Splunk logging API for iPhone developers, call SPLogger.  We’d love feedback, code contributions, suggestion.  The SPLogger API allows iPhone developers to log events in their application and have them go to Splunk Storm (www.splunkstorm.com), which is free for up to a GB of data. If you currently have no insight into how your app is being used, or by whom, this can come in handy, and of course you’ll have the full power of SPL, Splunk’s search language.

To get the SPLogger API, download it via either method:

Using Storm for Analytics

By using SPLogger, all events from all mobile devices are uploaded

» Continue reading

Poke at our API

With this tool:

$ splunk _internal call <relative rest path>
[-get:<param> <value>] ... [-post:<param> <value>] ...
[-method <http action>] [-multival] [-auth <user>:<pass>]

As mentioned in my previous post, exploring our endpoints is pretty simple to do, by pointing your browser at the Splunk management port. Actually making use of the endpoints requires more work, but this utility makes it easy to get started.

Restarting an input component is a handy example, such as restarting monitoring after editing inputs.conf by hand:

splunk _internal call /data/inputs/monitor/_reload
This is supported by the other components in /data/inputs, as well – browse there and look for the _reload links.

Parameters

  • get:foo bar – adds an HTTP GET parameter to the request, with name ‘foo’
» Continue reading

Talk to Splunk from WordPress

I wrote a WordPress plugin (tested for 2.5.1) that displays my most recent Google search terms in my sidebar. It was an experiment with using the Splunk REST API and the PHP SDK.

You can configure the widget from the Widgets page and it supports multiple instances with different configuration. Right now the actual search string is hardcoded because I’m doing some extra mangling to get the search terms the way I want anyway, but I’ll be adding that to the configuration options also. Eventually there will be a way to cache results so you don’t do the search each time the page is loaded.

Since there is still work to do to make it more generic, I haven’t uploaded …

» Continue reading