Using machine learning for anomaly detection research

Over the last years I had many discussions around anomaly detection in Splunk. So it was really great to hear about a thesis dedicated to this topic and I think it’s worth sharing with the wider community. Thanks to its author Niklas Netz in advance!

Obviously anomaly detection is an important topic in all core use case areas of Splunk, but each one has different requirements and data, so unfortunately there is not always an easy button. In IT Operations you want to detect systems outages before they actually occur and proactively keep your depending services up and running to meet your business needs. In Security you want to detect anomalous behavior of entities to detect potential indicators for breaches …

» Continue reading

Detect IoT anomalies and geospatial patterns for logistics insights

In part 1 of this blog series we spoke about how to turn sensor data into logistics insights. In this part we outline one approach for anomaly detection and enrich our sensor data with location information to discover geospatial patterns.

Anomalies? Find them with a few lines of SPL.

Anomaly detection can be tricky and implementations vary from simple thresholding and baselining to highly sophisticated approaches based on machine learning. In this example we leveraged the Splunk Machine Learning Toolkit to detect numeric outliers using a sliding window approach to check against multiples of the standard deviation in this time series to spot anomalies.

iot_blog_timechart_anomalies_chart

And that’s how the SPL looks like:

index="sensor"
| timechart span=1s avg(ax) as avx avg(ay) as

» Continue reading

Turn IoT sensor data into Operational Intelligence for logistics

The Internet of Things (IoT) wave may impact businesses and industry verticals differently but with the same potential: IoT opens new doors to interesting use cases that have immediate business impact and value. Splunk has delivered Operational Intelligence and Analytics in IT and Security for years, so why not apply Operational Intelligence and Analytics to IoT?

IoT_logistics_overviewReferring to the general definition of IoT we consider an object that is connected to the internet, in our case data coming from a sensor which measures acceleration. One use case I want to walk through here is not new to logistics, but a great example to show the value in IoT. As the diagram above depicts the globalized delivery of goods takes place …

» Continue reading