It’s cold outside – A year’s worth of data from my pellet heating unit!
Over a year ago I shared a blog about my efforts to monitor the pellet unit I use for heating my home. I wanted to share a report on how it went with access to a year’s worth of data and what new tuning steps I have added as a result. Remember – I’m an IT guy – not a pellet home heating system expert – but as I spoke to the experts they got really excited about what I found and how I could fine tune everything – so I thought I would share it with you so that you too can mess up the configuration of your home heater as well ;-).
A year’s worth of data…
Analyzing BotNets with Suricata & Machine Learning
Since the official rollout at the year’s. conf of the Machine Learning Toolkit(MLTK), Splunkers have been pursing some interesting use cases ranging from IT operations, planning, security and business analytics. Those use cases barely scratch the surface of what is possible with machine learning and Splunk. As an example, I will use the machine learning toolkit and data collected from Suricata to analyze botnet populations. This population analysis will be used to create a model for predicting the Mirai botnet based on network features.
Suricata is an open source threat detection engine, which can be run in passive mode for intrusion detection or inline for intrusion prevention. My lab environment is configured for intrusion detection, meaning Suricata will not …
Gatwick Airport lands passenger experience & operational efficiency with IoT, analytics and Splunk Cloud.
95% of passengers through security in 5 mins or less.
We’ve all been there, stuck at an airport, flight delayed, watching the departures board, trying to find somewhere to sit down and wanting to set off to where you’re going or just get home. Gatwick Airport, the busiest single runway airport in the world, processing up to 945 flights per day is striving to make this situation a thing of the past. I’m delighted to announce that they are using data from the Internet of Things and Splunk Cloud to improve the passenger experience and enhance operational efficiencies across the airport. Gatwick are using their machine data to deliver historic, real-time and predictive analytics to ensure a faster journey through the …
Dashboard Digest Series – Episode 4 – NFL Predictions
In Episode 4 we will take a look at the four downs of football. We used the Machine Learning Toolkit and more than a decade of NFL data to build models to make predictions during NFL games.
In order to make it quick and easy to plug in a scenario and visualize the most likely outcomes, we made a simple dashboard so editors at Sports Illustrated could try it out during a game. You may have seen the dashboard if you were watching CNN before the Super Bowl earlier this year:
The data contains a lot of fields…
Dashboard Digest Series – Episode 3
Welcome to Episode 3 of the Dashboard Digest series! At Splunk we love to eat our own dogfood so in this episode we will see a dashboard showing energy and water usage at Splunk headquarters in San Francisco! Additionally you’ll see a few new custom visualizations that became available for use in Splunk 6.4 as well as use of the Machine Learning Toolkit.
Purpose: Display and analyze building energy and water usage. Use machine learning to forecast energy usage, detect outliers and look for anomalies.
Splunk Version: Splunk 6.4 and above
Data Sources: Sensor data in JSON format coming from Aquicore devices.
Apps: Machine Learning Toolkit, Water Gauge Visualization, Calendar Heatmap Visualization
Summary of tips/tricks used:…
Dashboard Digest Series – Episode 2: Part Deux
Before moving on to the next episode 3 I decided to do a part two of Episode 2 – Waves! The reason being is two-fold. 1) Splunk Enterprise 6.5 was recently released and 2) Hurricane Matthew had quite the effect on some of these buoys/stations. See the original blog post here: Dashboard Digest Series – Episode 2
Purpose: Display meaningful statistics on NDBC buoy information in historical and real-time. Easily drilldown, aggregate and visualize data from 1000s of buoys transmitting information.
Splunk Version: Splunk 6.5 and above for table coloring
Data Sources: Polling NDBC RSS feed that produces JSON payload
Apps: Add-on for NDBC, Custom Cluster Map Visualization, Clustered Single Value Map Visualization, …
Reimagining IT at .conf2016
Last month, during the IT Ops Keynote at Splunk .conf2016, Splunk’s Chief Technology Advocate Andi Mann talked about the massive impact and opportunity created by digital transformation. Every industry, every business and every organization is experiencing the effects of digitization and dealing with an astounding rate of change. Whether it’s software-defined-everything, containerization, microservices or the world of the Internet of Things (IoT), digital transformation is everywhere. This newest evolution of IT is disrupting market leaders and upending entire industries – pushing every business to be a technology business. Digital transformation is also changing the technology we use as well as the way our teams connect, work and solve problems.
“New approaches to IT bring people, activities &
Analyzing the Mirai Botnet with Splunk
On September 20th, the largest Distributed Denial of Service attack ever recorded targeted security researcher Brian Krebs. This attack was made up of Internet of Things (IoT) devices such as cameras, wireless controllers and internet enabled devices peaking at 400,000 total. Now dubbed the “Mirai botnet”, these devices scanned the internet for devices running telnet and SSH with default credentials, infecting them and further propagating. The source code for the botnet has since leaked to GitHub, where further analysis is underway by security researchers.
During the infection time period, I happened to be running a honeypot and captured some infection attempts on my own system. Using Suricata and /var/log/secure.log I can correlate invalid login attempts associated with Mirai with malicious …
It’s a Family Affair…What’s up with the Family?
“Its a Family Affair“ Whenever I sing that Sly and the FamilyStone song around the house the kids just roll their eyes with disgust. I laugh and tell them not to ’dis on an old great tune. While singing it I often wonder how Sly’s lyrics would be different today considering the technical challenges facing parents trying to monitor their teenager’s online activities.
Fortunately, there are many great methods both paid and free that are available to parents these days. The good news is that no matter what your parenting style there is a tool that you can probably use to monitor your children’s online behavior. The bad news is there are just too many tools to choose from and …
I’m sensor-ing that the fourth industrial revolution is going to be data driven
I was lucky enough attend the IoT World conference this week in Berlin. Everyone who is anyone in Industrial IoT and the associated software industry was present. The list of speakers included Bosch, GE and Vodafone among many others.
During the course of the two days at the event I had a conversation with a robot (see below), I visited a pre-war ballroom and I received a cocktail from two juggling bar tenders! However the most memorable moment came during the key note speech from Professor Whalster, one of the founders of Industry 4.0 movement – which is alternatively known as the fourth industrial revolution.
In simplistic terms, Industry 4.0 is focussed on the “smart factory” i.e the computerisation of manufacturing. …