Splunking a Microsoft Word document for metadata and content analysis
The Big Data ecosystem is nowadays often abbreviated with ‘V’s. The 3Vs of Big Data, or the 4Vs of Big Data, even the 5Vs of Big Data! However many ‘V’s are used, two are always dedicated to Volume and Variety.
Recent news provides particularly rich examples with one being the Panama Papers. As explained by Wikipedia:
The Panama Papers are a leaked set of 11.5 million confidential documents that provide detailed information about more than 214,000 offshore companies listed by the Panamanian corporate service provider Mossack Fonseca. The documents […] totaled 2.6 terabytes of data.
This leak illustrates the following pretty well:
- The need to process huge volume of data (2.6 TB of data in that particular case)
- The need to
Splunking Microsoft Azure Audit Data
We recently made available a community-supported Splunk Add-on for Microsoft Azure, which gives you insight into Azure IaaS and PaaS. I am happy to announce that this add-on now includes the ability to ingest Azure Audit data. The idea behind Splunking Azure Audit logs is to be able to tell who did what and when and what events might impact the health of your Azure resources. In this blog post, I will detail what we are collecting, how to use the data, and what is coming next for the add-on.
What are we collecting?
This update adds a new modular input to your Splunk environment:
This modular input grabs data using the Azure Insights Events API.
Splunking Microsoft Azure Data
There are a lot of services in Microsoft Azure, and a lot of those services are producing machine data. Hal Rottenberg wrote a post covering several of these services and some ways to integrate Splunk with Microsoft Azure. We recently released a new cross-platform Azure add-on that consumes data for some IaaS and PaaS services. In this blog post, I will detail what we are collecting, how to use the data, and what is coming next for the add-on.
What are we collecting?
The add-on ships with three modular inputs:
- Azure Diagnostics – this input collects data from an Azure Storage account that contains virtual machine diagnostic information.
- Azure Website Diagnostics – this input collects server and application data for Azure
Announcing Splunk Enterprise in Microsoft Azure Marketplace
We are pleased to announce the release of Splunk Enterprise in Microsoft Azure Marketplace!
Now Azure customers can deploy and purchase Azure-certified Splunk Enterprise clusters in minutes, with the entire point-and-click workflow contained within their Azure portal.
This Bring-Your-Own-License offering on Azure IaaS, provides Splunk customers another platform for self-managed Splunk deployments in addition to on-premise and other public cloud deployment options.
What can Splunk Enterprise in Azure Marketplace do for you?
Our mission at Splunk is to make machine data accessible, usable and valuable to everyone. We strive to turn machine data into valuable insights in as little time as possible to help businesses in their journey towards operational intelligence:
Splunk Enterprise in Azure Marketplace enables and …
Splunk and Microsoft Azure – Intro and Resource Roundup
Note: the below article was written back in Dec 2014, but still gets a ton of hits and questions. Be sure to check out the Azure tag here on Splunk Blogs for the latest news.
We are often asked by customers about how Splunk can integrate with, or run in Microsoft’s Azure cloud platform. There’s actually a fair bit of information about this broad topic on splunk.com and elsewhere, but it can be a bit hard to find. This post will serve as an introduction to a few Azure …
Monitoring Network Traffic with Sysmon and Splunk
Every IT guy has a set of tools that they use every day. One of mine is sysinternals. It’s a set of Windows utilities made available by Microsoft that do a whole slew of things. You can install them with chocolatey (another in my toolset) or downloaded and unpacked from their website. If you use Windows and this toolset isn’t in your arsenal, maybe it’s time.
Back in August, I got a request from one of our engineers asking me if we had any plans to support the collection of Sysmon data. Sysmon is a Windows system service (yes, another agent) that logs system activity to the Windows Event Log. However, it places all the important stuff in the …
Splunk App for SharePoint goes Open Source
For about the last year, I’ve been working on an update to the Splunk App for SharePoint. But it isn’t the one you would expect. I’ve been working to open source the app. At the end of the day the best person to write an IT Operations app for Splunk is the person who is intimately involved in the running of the workload. Today, we are flicking the switch and opening up the project. We are allowing you to directly file bugs and feature requests; we are allowing you to submit code; and we are encouraging you to get involved in the project.
So, how can you do this. Firstly, you will want to have some sort of test environment. …
Splunk 6.2 Feature Overview: Perfmon Delocalization
Last week, I covered the XML Event Logs – an awesome feature that will reduce your data ingest, increase the fidelity of the data that is stored and allow us to work with localized data. Today, I want to discuss another localization feature – or at least a delocalization feature – perfmon.
Prior to Splunk 6.2, Windows perfmon was always collected localized. If you wanted the % Processor Time counter, you had to specify the localized version of this. If you were running on a french version of Windows, you would have to specify object=Processeur and counter=”% Temps Processeur” in both your inputs.conf and searches. Given that there are over 30 different localized versions of Windows, this really meant that …
Splunk 6.2 Feature Overview: XML Event Logs
We’ve been (rightly) criticized for a couple of things in recent years. Firstly, when you configure a Windows Event Log, it’s too big. This is because we combine the event log object with the message from the locale-specific DLL and that includes a bunch of common explanatory text. I don’t really need to know what a login really means (to the tune of 1K of data ingest) every time someone logs in, especially when these events are happening hundreds of times a minute. Secondly, our event log extractions are for US/English only. Got German Windows? Sorry – our extractions don’t work for that. Finally, we discard the additional data that is provided in the event log object. A primary example …
RDP to Windows Server from a Splunk Dashboard – Example Code
A while back, I wrote blog post explaining how to RDP to a Windows Server from a Splunk Dashboard. The steps involved the following:
- Create a Controller – this generates the .rdp file on the server and delivers it to the client.
- Create a custom endpoint in web.conf – this part enables url access to the controller created above.
All the nitty-gritty details were spelled out in the blog post. However, if you learn better by example (like I do), then there is a new GitHub repo that has a working example for you. In the …