Writing Actionable Alerts
Is your Splunk environment spamming you? Do you have so many alerts that you no longer see through the noise? Do you fear that your Splunk is losing its purpose and value because users have no choice but to ignore it?
I’ve been there. I inherited a system like that. And what follows is an evolution of how I matured those alerts from spams to saviors.
Let it be known that Splunk does contain a number of awesome search commands to help with anomaly detection. If you enjoy what you read here, be sure to check them out since they may simplify similar efforts. http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Commandsbycategory#Find_anomalies
Stage 1: Messages of Concern
Some of the first alerts created are going to be searches …
My Splunk Origin Story
A World Without Splunk
In my pre-Splunk days, I spent significant time leading the vision for standards and automation in our company’s large distributed IBM WebSphere Network Deployment environment. Even though we used standard build tools and a mature change process, significant entropy and deviations were introduced into the environment as a product of requirements for tuning, business, infrastructure, security, and compliance.
As a result, we were unable to recognize the scope of impact when it came to security vulnerabilities or violations with 3rd party compliance. Even worse for us, we spent way too many staff-hours trying to replicate issues between production and quality assurance environments because we had no easy way to recognize the contributing configuration differences.
It’s a Bird, It’s a …
What is Operational Intelligence? Real-World Examples
Having run through the four levels of Operational Intelligence (level 1, level 2, level 3, level 4) I thought it made sense to end this blog series with some customer examples. I’ve tried to pick an example from each industry and try to cover IT Operations, Security and Customer Experience. I’ve also included a link to a case study or press release where possible.
UniCredit uses Splunk Enterprise for real-time insights into multiple terabytes of operational data and to monitor key business metrics. Proactive incident management has resulted in about 40% of incidents managed before becoming evident to end users, while problem solving and troubleshooting time has been reduced by 70%. UniCredit has improved …
What is Operational Intelligence? Level 4
We’re at level 4 of Operational Intelligence adoption (feel free to catch up level 1, level 2 or level 3) where data gives an organisation real-time insight in order to make key business and IT decisions.
At level 4, Operational Intelligence will be delivering value to many parts of a company and giving real-time analytics to support business decisions and planning. The amount of data may well be “big data” from potentially hundreds of different sources used for multiple use cases (IT operations, security and customer experience). The machine data generated by an organisation will be augmented with other data – relational sources, mobile devices and social network information. Operational Intelligence level 4 enables a data driven approach to transforming a business. …
What is Operational Intelligence? Level 3
In part one of this blog series we spoke about Operational Intelligence adoption level 1 which is focused on searching and investigating your data. In part two we described Operational Intelligence adoption level 2 that enables organisations to become more proactive and be alerted to patterns in data in real-time.
Operational Intelligence adoption level 3 is where real-time analytics starts to play an important part in visualising your IT, security or customer focused operations. The amount and types of data that organisations analyse is typically larger and the intelligence will be accessed by a wider range of people. Let’s get into more detail on adoption level 3:
You’re likely to have delivered some form of real-time reporting and analytics from …
What is Operational Intelligence? Level 2
In part one of this blog series we went through Operational Intelligence level 1 from the perspective of IT Operations, Security and Customer Experience.
As organisations discover the benefits of searching and investigating their data, they start to see how they can become more proactive and start to see issues before they arise. This typically means seeing a pattern of data that indicates something outside a “normal” baseline and setup notifications and alerts when unusual collections of events start to happen. This is Operational Intelligence adoption level 2:
You’ve moved beyond collecting data and reacting to issues by searching for the cause. You’re likely to be monitoring most, if not all of your IT and application estate, in …
What is Operational Intelligence? Level 1
It is probably a bit late to say “Happy New Year” so maybe “Happy New Blogging Year” might be more appropriate. As part of last year’s research with analyst company Quocirca, we looked into the fastest growing concerns for European business and the role Operational Intelligence plays in addressing them. The three main concerns from the research were security threats, data chaos from IT systems and poor customer experience. The idea of this blog series was to explain the four levels of Operational Intelligence and how they map to each of the three main concerns. To give you an introduction to the four adoption levels of OI, it looks a little something like this:
Let’s get started with OI level …
IT Service Intelligence – A Bridge between Business and Technical Teams
Recently, after working with a Business Team and the Technical Team that supports them at one of our customers, we had an opportunity to witness, first-hand, the struggles each of us in IT Operations has felt at some point. In the words of a wise man, they were experiencing “Mutual Mystification.” The Business Team was concerned with a Product Line and multiple business processes. The Technical Team was trying to translate the technical details of the underlying micro services and how they were related to the Product Line. Within the first 15 minutes, both teams were becoming frustrated and didn’t feel like they were communicating well. This led to a consensus that the meeting was not going to achieve the results …
Splunk Delivers Real-Time Operational Intelligence to Newly Announced AWS IoT Service
The big news of the day is that AWS is officially in the Internet of Things business. This is an announcement that is bound to make a major impact on the IoT, and is exciting news for those who are building IoT solutions as they can now take advantage of the benefits AWS can provide in terms of time-to-value, security, and scalability for device to cloud and cloud to device infrastructure.
This is also great news for Splunk’s IoT team. Our existing partnership with AWS provided a great starting point for making Splunk Enterprise AMIs and Splunk Cloud an easy to deploy and use solution for the massive amounts of machine data bound to be created by AWS IoT applications. …
Cheers to .conf2015 with Three Clicks and a Beer
Tuesday was the kickoff of .conf2015: The 6th Annual Splunk Worldwide Users’ Conference in Las Vegas and it was incredible. After months of preparation, we were ready to hit the stage for the keynote and show the audience – our customers – how much we appreciate their loyalty, their innovation, and their inspiration. The room was packed. The staging was absolutely impressive. The place was buzzing. I was, and still am, in awe of the amount of work, preparation, and production needed to pull off an event of this scale. It’s just one more example of why I am so thrilled to be part of this team.
I was the third speaker in an impressive lineup of Splunkers …