Send data to Splunk via an authenticated TCP Input

Wow , my second blog in 24 hrs about Protocol Data Inputs(PDI) , but sometimes you just infected with ideas and have to roll with it.

So my latest headbump is about sending text or binary data to Splunk over raw TCP and authenticating access to that TCP input.Simple to accomplish with PDI.

Setup a PDI stanza to listen for TCP requests

PDI has many options , but for this simple example you only need to choose the protocol(TCP) and a port number.

Screen Shot 2016-07-30 at 3.31.08 PM

Declare a custom handler to authenticate the received data

You can see this above in the Custom Data Handler section.I have declared the handler and  the authentication token that the handler should use via a JSON properties …

» Continue reading

Sending binary data to Splunk and preprocessing it

A while ago I released an App on Splunkbase called Protocol Data Inputs (PDI)  that allows you to send text or binary data to Splunk via many different protocols and dynamically apply pre processors to act on this data prior to indexing in Splunk. You can read more about it here.

I thought I’d just share this interesting use case that I was fiddling around with today. What if I wanted to send compressed data (which is a binary payload) to Splunk and index it ? Well , this is very trivial to accomplish with PDI.

Choose your protocol and binary data payload

PDI supports many different protocols , but for the purposes of this example I just rolled  a …

» Continue reading

True Machine Learning is finally here for all to Leverage

In this short post I want to hit 3 simple points.

  • Why has true machine learning been so difficult to provide to the masses?
  • Why is machine learning not simply statistical models?
  • What type of organization has the power to bring machine learning to the masses?

The simple reason it took so long to bring machine learning from the theoretical to the everyday is that it is hard. No scratch that it is really – really hard to do at scale and price point where every organization can leverage the innate power of machine learning.

Think about it, you have this layer of intelligence over all your machine data that is constantly on the watch for unusual behavior and anomalies. …

» Continue reading

If your plants could speak to you, what would they say?


I’m pretty sure mine would say “Hey Bozo, thanks for drowning me to death” or “Must… have… water… What is this, the Sahara?” Oh, and also “I hate it here, what’s it take to get some morning sun?”

I decided it was time to apply my inner nerd to reduce my plants suffering. That and happier plants mean a happier fiancé. Enter Splunk! The goal was:

  1. Keep track of moisture level in the soil.
  2. Determine best location for light intake.
  3. Combine current weather data, future forecasts and 1 and 2 above to create some machine learning models that predict when is best to water. (I’m still working on this part)

I shall call it… Operational Plantelligence! When first said aloud, …

» Continue reading

Best Practices in Protecting Splunk Enterprise

Splunk EnterpriseSplunk Enterprise helps companies collect, analyze, and act upon the data generated by their technology infrastructure, security systems and business applications. Customers use Splunk software to achieve operational visibility into critical information technology assets and drive operational performance and business results.

Splunk Apps enhance and extend the Splunk platform and deliver a user experience tailored to typical tasks and roles. Most customers make use of one or more of the 1000+ Apps available in Splunkbase.

While end-users are the main consumers of Apps, App installation requires full administrator access. We strongly discourage customers from granting this access to any user other than designated administrators.

Beyond restricting admin privileges, we recommend adopting the standard deployment and operation practices described briefly …

» Continue reading

2016 State of DevOps Report: Release 200x and Recover 24x Faster

D1339Sponsor_Graphics_600x300I am happy to announce that Splunk is a proud sponsor of  2016 State of DevOps 2016 Report authored by our technology partner Puppet and DevOps Research and Assessment (DORA). With more than 4,600 responses from IT professionals around the world, this report is one of the most comprehensive in the industry. It examines important trends in the DevOps community today. Key results include:

  • High-performing organizations deploy 200x more frequently than low performers, with 2,555x faster lead times and 24x faster recovery times
  • High performers spend 22% less time on unplanned work and rework, and 29% more time on new work than low performers
  • High performers spend 50% less time remediation security issues than low performers
  • Employees in
» Continue reading

DevOps Metrics: Measuring Business Impact

A few weeks ago, I attended DevOpsDays Seattle. As a big fan of Open Space, I was happy when my session proposal on DevOps metrics was accepted. And as it turns out, this is an important subject for DevOps community, as my colleague, Splunk Chief Technology Advocate, Andi Mann also noted in his blog. Below are some of the highlights from those discussions among 30+ DevOps practitioners.seattle 3

Measuring Business Impact

Since the audience was from a variety of organizations of different sizes and maturity, it was great that a common theme/metric bubbled up: knowing and measuring the business impact of DevOps activities. The first step is to identify a business or customer success goal. Other relevant parameters stem …

» Continue reading

Splunking Euro2016 – an analytics approach to who’s going to win

UPDATED 8th July for France vs Portugal

So I’m having mixed results. I said Germany would win. Clearly wrong after they lost to France last night. I think that says more about my analysis than Splunk. It also shows the importance of having the most upto date, real-time data and not basing decisions on a 2012 data set! Nevertheless as Winston Churchill said “Success is stumbling from failure to failure with no loss of enthusiasm” so I thought I’d see what the data says for the final of France vs Portugal.

First up goals scored and conceded in 2012:


Portugal had the edge in 2012. So far in Euro 2016, France have been pretty good defensively and in terms of …

» Continue reading

Team Agility with The Splunk Platform

In continuing my DevOps series I talked about how my journey at Splunk sparked my drive to help our customers get a better understanding of application delivery cycle. I briefly talked about DevOps methodology and basic concepts. But the most important element is how your team responds to DevOps–driven workflow.

DevOps teams use numerous types of project management tools while many companies are looking to move into a Scrum process flow or become a full agile shop. Tools like Atlassian’s JIRA, Version One, and Rally enable teams to see the value of their work by tracking project tasks in Sprints that can represents various work periods. Teams that follow this type of flow have a better understanding of items that …

» Continue reading

One source, many use cases: How to deliver value right away by addressing different IT challenges with Splunk – Part 2

Do you remember this piece of raw data:


I hope so, it was on the blog only last week … 😉

Today, let’s focus on the value we can extract and how we’ll be able to address some of the IT challenges related to the company strategy.

IT Ops

What kind of information would be relevant for the application manager?

I am sure he would be interested by:

  • Number of transactions during the last X minutes and the trend
  • Number of transactions in errors during the last X minutes and if this number is growing compared to the last Y minutes
  • How long a transaction takes to complete for each customer
  • A geographic distribution of the transactions

“What? You said …

» Continue reading