Wait, what – a youtube video for my app!?
At Splunkbase we are constantly striving to improve the experience for our users – whether it’s the app-discovery process for a Splunk admin/user, or the app-submission and management experience for our developers. We’ve been busy making changes over the last few months, and I thought this would be a good time to cover some of the more important changes we’ve made recently.
There was a lot of backend engineering work done to spruce up the infrastructure, the API, and search results relevancy – changes that are not always apparent to an end-user of Splunkbase. However, in this post I will talk about some user-facing features we recently added with the goal of improving the experience for our developer community. These features will allow you to …
Send JSON objects to HTTP Event Collector using our .NET Logging Library
Recently we shipped a bunch of logging libraries at the same time our new HTTP Event Collector hit the streets: http://blogs.splunk.com/2015/10/06/http-event-collector-your-direct-event-pipe-to-splunk-6-3/
One of the questions I’ve heard from customers using the libraries, is “Can I send JSON objects with the .NET logging library?”
Yes, you can. To do it, you need to use our Splunk.Logging.Common library which our other loggers depend on. Interfaces like TraceListener were designed for sending strings not objects.
For example TraceSource has a TraceData method which accepts objects and which it appears should work. However (at least based on my testin)g the objects are serialized to strings and then passed on as such to the listeners. Thus by the time we get it we …
Join Splunk at the Emirates Travel Hackathon next weekend!
Splunk is pleased to be sponsoring the Emirates Travel Hackathon next weekend, and we want to see you there! The event is taking place Nov 7th in San Francisco, and is open to all who want to participate in a great event with real prizes. Come out, learn something new, and solve challenging problems in the realm of travel! What should you build? Here’s what Emirates is looking for (from the FAQ page):
The hack should revolve around the experience of traveling. There are many websites and applications that focus on the logistics of travel – scheduling flights, reserving seats, booking hotels, etc. However, we’re looking for apps that help people experience the excitement of travel to the fullest extent, so get creative
Splunking Sensor Data with Arduino and HTTP Event Collector
It’s been (relatively) chilly in the SF office the last few weeks, but given “how I feel” is rather subjective I figured it would be an excellent chance to both gather some empirical evidence, and try out the new Splunk HTTP event collector! In this post I will walk you through setting up an Arduino with an ethernet shield and temperature sensor to log data directly to Splunk.
Ethernet Shield (the older model but new should work too)
Arduino Sketch (code)
Splunk 6.3 (even the free one!)
Wiring It Up (Fritzing Diagram)
Setting Up Splunk HTTP Event Collector
First things first, lets set up the HTTP Event Collector to be able …
Custom Message Handling and HEC Timestamps with the Kafka Modular Input
Custom Message Handling
If you are a follower of any of my Modular Inputs on Splunkbase , you may see that I employ a similar design pattern across all of my offerings. That being the ability to declaratively plug in your own parameterizable custom message handler to act upon the raw received data in some manner before it gets output to Splunk for indexing. This affords many benefits :
- Many of my Modular Inputs are very cross cutting in terms of the numerous potential types and formats of data they will encounter once they are let loose in the wild. I can’t think of every data scenario. An extensibility design allows the user and community to be able to customize the
Splunk at Dynatrace PERFORM
This week, Splunk will be participating at Dynatrace PERFORM – the annual users event for Dynatrace APM users. Not only is Dynatrace the largest APM vendor by market share, we know that many people are getting value by connecting Dynatrace APM with Splunk. The Dynatrace APM App for Splunk has nearly 1,500 downloads!
We’ll be at Dynatrace PERFORM largely to share what exactly Splunk is with attendees and how it can complement the capabilities found in Dynatrace’s products.
In addition to the “lightning talks” we’ll be delivering throughout Wednesday and Thursday, I’m honored to be on a New Stack panel on Thursday October 15 at 4:30pm, along with representatives from AWS, NGINX, Ansible and NodeSource.
Achieving scale with the Kafka Modular Input
A hot topic in my inbox over the recent months has been how to achieve scalability with the Kafka Modular Input , primarily in terms of message throughput. I get a lot of emails from users and our own internal Splunk team about this , so rather than continuing to dish out the same replys , I thought I’d just pen a short blog to share some tips and tricks.
So let’s start off with this simple scenario :
- a single instance of Splunk 6.3
- downloaded and installed the freely available Kafka Modular Input from Splunkbase
These are the scaling steps that I would try in order.
Enable HTTP Event Collector output
With the recent release of Splunk 6.3 , …
Scheduled Export of Indexed Data
I’m really enjoying playing with all the new Developer hooks in Splunk 6.3 such as the HTTP Event Collector and the Modular Alerts framework. My mind is veritably fizzing with ideas for new and innovative ways to get data into Splunk and build compelling new Apps.
When 6.3 was released at our recent Splunk Conference I also released a new Modular Alert for sending SMS alerts using Twilio, which is very useful in it’s own right but also a really nice simple example for developers to reference to create their own Modular Alerts.
But after getting under the hood of the Modular Alerts framework, this also got me thinking about other ways to utilise Modular Alerts to fulfill other use …
Notes From Splunk .conf 2015 Day Two
The Search party last night was a blast, but today it was back to business. And Day 2 of the global Splunk user group, .conf2015, was another excellent day.
I started with some good mates from the industry analyst community, talking Splunk IT Service Intelligence (ITSI) over breakfast. I gained intriguing insights into our customers and our market, and came away with all sorts of possible new use cases for ITSI.
But as Steve Jobs said, innovation sometimes it means saying ‘no’ to a thousand good ideas, so for now we are going to focus on fulfilling the enormous early demand from our customers for POCs. Still, we are always looking for new ideas from our customers and partners (and analysts too!), …
Turbo charging Modular Inputs with the HEC (HTTP Event Collector) Input
HTTP Event Collector (HEC)
Splunk 6.3 introduces a new high performance data input option for developers to send event data directly to Splunk over HTTP(s). This is called the HTTP Event Collector (HEC).
In a nutshell , the key features of HEC are :
- Send data to Splunk via HTTP/HTTPS
- Token based authentication
- JSON payload grammar
- Acknowledgment of sent events
- Support for sending batches of events
- Keep alive connections
A typical use case for HEC would be a developer wanting to send application events to Splunk directly from their code in a manner that is highly performant and scalable and alleviates having to write to a file that is monitored by a Universal Forwarder.
But I have another use case …