Splunking Microsoft Azure Data
There are a lot of services in Microsoft Azure, and a lot of those services are producing machine data. Hal Rottenberg wrote a post covering several of these services and some ways to integrate Splunk with Microsoft Azure. We recently released a new cross-platform Azure add-on that consumes data for some IaaS and PaaS services. In this blog post, I will detail what we are collecting, how to use the data, and what is coming next for the add-on.
What are we collecting?
The add-on ships with three modular inputs:
- Azure Diagnostics – this input collects data from an Azure Storage account that contains virtual machine diagnostic information.
- Azure Website Diagnostics – this input collects server and application data for Azure
Another Update to Keyword App
It’s been three years since I first released the relatively simple Keyword app on Splunkbase and wrote an initial blog entry for it describing it followed by an updated entry. In summary, the Keyword app is a series of form search dashboards designed for Splunk 6.x and later that allow a relatively new user to type in keywords (e.g., error, success, fail*) and get quick analytical results such as baselines, prediction, outliers, etc. Splunk administrators can give this app to their users as is, use the app as a template to write their own keyword dashboards, or take the searches in the app to create new views.
For this update, I’ve used, fellow Splunker, Hutch’s icons to update the …
Announcing Splunk Enterprise in Microsoft Azure Marketplace
We are pleased to announce the release of Splunk Enterprise in Microsoft Azure Marketplace!
Now Azure customers can deploy and purchase Azure-certified Splunk Enterprise clusters in minutes, with the entire point-and-click workflow contained within their Azure portal.
This Bring-Your-Own-License offering on Azure IaaS, provides Splunk customers another platform for self-managed Splunk deployments in addition to on-premise and other public cloud deployment options.
What can Splunk Enterprise in Azure Marketplace do for you?
Our mission at Splunk is to make machine data accessible, usable and valuable to everyone. We strive to turn machine data into valuable insights in as little time as possible to help businesses in their journey towards operational intelligence:
Splunk Enterprise in Azure Marketplace enables and …
Excelling with Excel in Splunk
if you didn’t already know that you can heavily customize Splunk through our open developer framework you should check it out. You can even develop and introduce new search commands. This particular blog illustrates this with an example where business people wanted to have excel files with reports mailed to them from Splunk.
You might already know that Splunk enables you to connect Excel directly to Splunk with the ODBC Connector for Windows, as well as enabling you to export with outputcsv a csv file. Dominique from Helvetia Insurance has developed a Splunk TA that is freely available on Splunkbase which allows you to import, export and e-mail data in XLS format.
Dominique notes that, “If tabular reports …
Remote Images Retrieval With Splunk Using Custom Command “getimage.py”
Every once in a while my customers ask for a functionality that is not natively supported by Splunk. Out of the box Splunk is a very capable platform, however, there are certain tasks Splunk is not designed for. But that never stops a Splunker from finding a solution! The use-case I am about to discuss in this blog is an example of that: The customer owns large chain of pharmacies across the country, the bulk of the stores transactions end up in Hadoop Data Lake; the customer wants to use Hunk/Splunk to visualize and analyze the massive amount of information collected, which is something Hunk can do easily. The challenge came about when I was asked if Splunk could show …
Splunk Add-on > Where’s That Command – Converting a Field’s Hexadecimal Value to Binary
When looking through Splunk’s Search Reference Manual, there are a ton of search commands with their syntax, descriptions, and examples. After all, if Splunk is the platform for machine data, there needs to be an extensive list of commands, functions, and references that guide Splunkers through the Search Processing Language (SPL). But one would think that we had everything covered, right? Well, almost….
I have a couple of great customers from the Houston, Texas area to thank for this. Gabe and Andrew (you know who you are) are not only strong Splunkers, but frequent the Splunk Houston User Group (SHUG) meetings and are always looking for ways to expand their use of Splunk as well …
An Hour of Code with Splunk
The Hour of Code is a global effort to educate children in more than 180 countries with as little as one hour of computer science. Held as part of Computer Science Education Week (December 7-13), the most recent Hour of Code included more than 198,473 events around the world. And this year, several Splunkers taught sessions in events across the country.
Here in the Seattle Area, Shakeel Mohamed, one of our engineers, taught sessions on Lightbot and Minecraft at Rainier View Elementary School, and I had the pleasure of teaching approximately 150 students at Ingraham High School an hour about log / time-series data and how to mine it with Splunk. The courses are a challenging mix of students …
Splunk in Space: Splunking Satellite Data in the Cloud
This year a Team of Splunkers attended the ESA App Camp 2015 in lovely Frascati, Italy. The topic of this year’s challenge was:
“There are thousands of ways to enrich apps with data from space – what’s yours?”
The Splunk team featured Robert Fujara and Philipp Drieger alongside with camp participants Claire Crotty and Anthony Thomas. Together the team created a mobile web app that accessed a Splunk Cloud instance to analyze geolocation-based satellite data and inform users about different environmental indicators across Europe. Users can input their preferences in terms of living environment and based on different indicators they then receive recommendations on which city or region would suit them best.
The key data sources for this project…
Splunk Logging Driver for Docker
With Splunk 6.3 we introduced HTTP Event Collector which offers a simple, high volume way to send events from applications directly to Splunk Enterprise and Splunk Cloud for analysis. HTTP Event Collector makes it possible to cover more cases of collecting logs including from Docker. Previously I blogged on using the Splunk Universal Forwarder to collect logs from Docker containers.
Today following up on Docker’s press release, we’re announcing early availability in the Docker experimental branch of a new log driver for Splunk. The driver uses the HTTP Event Collector to allow forwarder-less collection of your Docker logs. If you are not familiar yet with the Event Collector check out this blog post.
You can get the new Splunk Logging …
Using Splunk Archive Bucket Reader with Pig
This is part II in a series of posts about how to use the Splunk Archive Bucket Reader. For information about installing the app and using it to obtain jar files, please see the first post in this series.
In this post I want to show how to use Pig to read archived Splunk data. Unlike Hive, Pig cannot be directly configured to use InputFormat classes. However, Pig provides a Java interface—LoadFunc—that makes it reasonably easy to use an arbitrary InputFormat with just a small amount of Java code. A LoadFunc is provided with Splunk Archive Bucket Reader: com.splunk.journal.hive.JournalLoadFunc. If you would prefer to write your own, you can find more information here.
Whereas Hive closely resembles a …