Detect IoT anomalies and geospatial patterns for logistics insights
In part 1 of this blog series we spoke about how to turn sensor data into logistics insights. In this part we outline one approach for anomaly detection and enrich our sensor data with location information to discover geospatial patterns.
Anomalies? Find them with a few lines of SPL.
Anomaly detection can be tricky and implementations vary from simple thresholding and baselining to highly sophisticated approaches based on machine learning. In this example we leveraged the Splunk Machine Learning Toolkit to detect numeric outliers using a sliding window approach to check against multiples of the standard deviation in this time series to spot anomalies.
And that’s how the SPL looks like:
| timechart span=1s avg(ax) as avx avg(ay) as
Turn IoT sensor data into Operational Intelligence for logistics
The Internet of Things (IoT) wave may impact businesses and industry verticals differently but with the same potential: IoT opens new doors to interesting use cases that have immediate business impact and value. Splunk has delivered Operational Intelligence and Analytics in IT and Security for years, so why not apply Operational Intelligence and Analytics to IoT?
Referring to the general definition of IoT we consider an object that is connected to the internet, in our case data coming from a sensor which measures acceleration. One use case I want to walk through here is not new to logistics, but a great example to show the value in IoT. As the diagram above depicts the globalized delivery of goods takes place …
SplunkZero, delivering value with Splunk at Splunk
I want to introduce you to our internal Splunk platform, SplunkZero. I’ll go into some detail on the philosophy of how we chose to deploy Splunk at Splunk, but what I hope to do is kick start the conversation about how we gain value with our own products.
A little bit about myself, in the 5+ years I’ve been here at Splunk, I have worked in both marketing and IT orgs and am excited to now be leading the SplunkZero team. I am passionate about our products and love seeing how excited our customers get when the talk about how they leverage Splunk.
The name SplunkZero came out of a request from our markets group that IT be driving internal …
If your plants could speak to you, what would they say?
I’m pretty sure mine would say “Hey Bozo, thanks for drowning me to death” or “Must… have… water… What is this, the Sahara?” Oh, and also “I hate it here, what’s it take to get some morning sun?”
I decided it was time to apply my inner nerd to reduce my plants suffering. That and happier plants mean a happier fiancé. Enter Splunk! The goal was:
- Keep track of moisture level in the soil.
- Determine best location for light intake.
- Combine current weather data, future forecasts and 1 and 2 above to create some machine learning models that predict when is best to water. (I’m still working on this part)
I shall call it… Operational Plantelligence! When first said aloud, …
Accenture and Splunk Form Alliance to Deliver Transformative Business Solutions
Strategic companies know that they can improve business outcomes by mining the vast amounts of application and operational data from back-end systems, call centers, web traffic and more. Today we announced a new partnership between leading global systems integrator Accenture and Splunk that integrates Splunk products and cloud services into Accenture’s application services, security and digital offerings. The alliance between Accenture and Splunk will enable customers to mine and analyze machine data and identify trends and opportunities that were previously difficult to detect.
Accenture is integrating Splunk’s platform for Operational Intelligence, including Splunk Enterprise, Splunk Cloud, Splunk Enterprise Security, Splunk User Behavior Analytics and Splunk IT Service Intelligence, into its world-class application and security service offerings.
Accenture Technology Services has …
Announcing Splunk Enterprise in Microsoft Azure Marketplace
We are pleased to announce the release of Splunk Enterprise in Microsoft Azure Marketplace!
Now Azure customers can deploy and purchase Azure-certified Splunk Enterprise clusters in minutes, with the entire point-and-click workflow contained within their Azure portal.
This Bring-Your-Own-License offering on Azure IaaS, provides Splunk customers another platform for self-managed Splunk deployments in addition to on-premise and other public cloud deployment options.
What can Splunk Enterprise in Azure Marketplace do for you?
Our mission at Splunk is to make machine data accessible, usable and valuable to everyone. We strive to turn machine data into valuable insights in as little time as possible to help businesses in their journey towards operational intelligence:
Splunk Enterprise in Azure Marketplace enables and …
Writing Actionable Alerts
Is your Splunk environment spamming you? Do you have so many alerts that you no longer see through the noise? Do you fear that your Splunk is losing its purpose and value because users have no choice but to ignore it?
I’ve been there. I inherited a system like that. And what follows is an evolution of how I matured those alerts from spams to saviors.
Let it be known that Splunk does contain a number of awesome search commands to help with anomaly detection. If you enjoy what you read here, be sure to check them out since they may simplify similar efforts. http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Commandsbycategory#Find_anomalies
Stage 1: Messages of Concern
Some of the first alerts created are going to be searches …
My Splunk Origin Story
A World Without Splunk
In my pre-Splunk days, I spent significant time leading the vision for standards and automation in our company’s large distributed IBM WebSphere Network Deployment environment. Even though we used standard build tools and a mature change process, significant entropy and deviations were introduced into the environment as a product of requirements for tuning, business, infrastructure, security, and compliance.
As a result, we were unable to recognize the scope of impact when it came to security vulnerabilities or violations with 3rd party compliance. Even worse for us, we spent way too many staff-hours trying to replicate issues between production and quality assurance environments because we had no easy way to recognize the contributing configuration differences.
It’s a Bird, It’s a …
What is Operational Intelligence? Real-World Examples
Having run through the four levels of Operational Intelligence (level 1, level 2, level 3, level 4) I thought it made sense to end this blog series with some customer examples. I’ve tried to pick an example from each industry and try to cover IT Operations, Security and Customer Experience. I’ve also included a link to a case study or press release where possible.
UniCredit uses Splunk Enterprise for real-time insights into multiple terabytes of operational data and to monitor key business metrics. Proactive incident management has resulted in about 40% of incidents managed before becoming evident to end users, while problem solving and troubleshooting time has been reduced by 70%. UniCredit has improved …
What is Operational Intelligence? Level 4
We’re at level 4 of Operational Intelligence adoption (feel free to catch up level 1, level 2 or level 3) where data gives an organisation real-time insight in order to make key business and IT decisions.
At level 4, Operational Intelligence will be delivering value to many parts of a company and giving real-time analytics to support business decisions and planning. The amount of data may well be “big data” from potentially hundreds of different sources used for multiple use cases (IT operations, security and customer experience). The machine data generated by an organisation will be augmented with other data – relational sources, mobile devices and social network information. Operational Intelligence level 4 enables a data driven approach to transforming a business. …