Splunk DB Connect 3 Released
Splunk DB Connect 3.0 is a major release to one of the most popular Splunk add-ons. Splunk DB Connect enables powerful linkages between Splunk and the structured data world of SQL and JDBC. The major improvements of this release are:
- Performance improvement. Under similar hardware conditions and environment, DB Connect V3 is 2 to 10 times faster than DB Connect V2, depending on the task.
- Usability improvement. A new SQL Explorer interface assists with SQL and SPL report creation.
- Improved support for scripted configuration, via reorganized configuration files and redesigned checkpointing system. Note that rising column checkpoints are no longer stored in configuration files.
- Stored procedures support
Using DB Connect With SQLite
The system uses a SQLite backend, so of course DB Connect was our first stop. This app uses the ubiquitous JDBC interfaces to provide access to all sorts of structured data sources, and it provides all sorts of enterprise grade options. However, we were able to skip over a lot of complexity for our use case.
First we needed a JDBC driver, and a little bit of web searching turned one up which worked on the first try.
Enabling JMX in WebSphere Application Server
Using Splunk to collect data from disparate sources is remarkably easy; but sometimes, making those sources emit data can take a bit more effort. Here’s some quick notes on making IBM WebSphere Application Server speak Java Management Extensions so that the Splunk Add-on for Java Management Extensions and Splunk Add-on for IBM WebSphere Application Server can be used to gather data. This isn’t comprehensive documentation for all possible scenarios, but rather some notes that we gathered in the development process.
Of course, the first step is to log on to the WebSphere Admin Console. Use the menu on the left to navigate to Servers -> Server Types -> WebSphere Application Servers. Click on the Application Server instance that you want to configure …
Relating Add-ons to CIM
Something we’ve been interested in for a while now is tools to help you see whether a model is being populated or not. For instance, the latest version of the Splunk App for Enterprise Security includes a nice Content Profile Audit dashboard that compares the knowledge objects provided in the Enterprise Security app to the data models those objects require.
Similarly, we also want to be able to look at a data model and ask which Add-ons are trying to prepare data for it. Thanks to the efforts of some intrepid folks in our Education team (Lincoln Bowser and Bob Walden), here’s a couple of reports that should be helpful. The reports query local configuration via REST so they’re cross-platform, and they leverage …
Notes on Splunk CIM
So you want to work with the Splunk Common Information Model, and you’re not sure where to start… developers first working with the CIM and Add-ons are sometimes confused by its minimalist design, particularly if they’re familiar with the broadly used Desktop Management Task Force CIM. Here’s some notes on the CIM’s design that hopefully will help clear things up. First, we’ll look at how it’s used, and then we’ll talk about why the Splunk CIM is designed the way that it is.
The Splunk CIM describes concepts via tags rather than entities via database columns, and the first thing to understand when you’re trying to work with it is the event type. Events are the raw material …
Machines, People, and Categories, Oh My!
Let’s say you’re working with Enterprise Security and you need to figure out how to put more devices into the asset and identity correlation framework. Here are some resources to get you started!
There are two useful types of data to integrate: lists of assets or identities, and attributes of assets or identities. In both cases, it may also be interesting to enable ad hoc, real-time queries of your data source for individual terms.
A list can be dumped from a directory, systems management tool, asset discovery system, or the like. These are typically accessed via DB Connect or Splunk Support for Active Directory. Other ways to get at this data include modular inputs to query web-based APIs. …
Custom Threat Feed integration with Enterprise Security
Threat intel feeds are a good way to add security context to your Splunk data with IP addresses, domain/host names or files. These feeds are generally accessible via some manner of web requests. Splunk Enterprise Security App has a Unified Threat Management framework for integrating threat intelligence feeds that makes these integrations easy . If the threat content you need to use is easy to download, you should be able to simply use the Configure -> Data Enrichment -> Threat Lists -> New form in the ES product.
But sometimes, a feed provider may require a number of steps before we can get the actual feed. Here’s how to handle a more difficult integration easily, using Symantec DeepSight’s threat feed …
Building Technology Add-ons
Happy New Year!
Following on Dennis Bourg’s post about using event generation, I’d like to post some of my notes about planning and building a technology add-on for use with Splunk. As we all know, getting data into Splunk is remarkably easy — we’re going to focus here on mapping the data to the Common Information Model so that it will be easy to use from other applications.
How do I analyze a data source for a TA?
First, we want to identify the data source and make sure that we understand how that data will be input.
- Search for the latest product guide. Go through the product guide to understand the product’s features and functionality.
- Try to understand
Read-only database connections
Version 1.1.1 of the Splunk DB Connect Add-on is now available on our community site, and there’s a great new option for managing your users’ database access. Let’s walk through how to grant selected users permissions to query only certain databases and have the option to restrict connections to read-only mode.
First, we’ll set up a test platform using Oracle’s MySQL.
* Install a MySQL database and a full JRE. Note that you don’t need to get JDBC drivers for MySQL, though you might have to download separate drivers for some database platforms. You might also want MySQL Workbench if you’re not comfortable with the command line.
* Upload the World sample database into your database.
* Create a …
Tuning Enterprise Security correlation searches
Here’s a nifty ES tuning tip that you might enjoy. We’ll be using some handy macros that are documented at Working_with_Notable_Events_from_Search, if you’d like to read up on the background.
What’s the most expensive, valuable, and constrained resource in a security team?
How many security analysts are there?
| `notable_owners` | stats count | eval sec_analysts=(count-1)
How long does it take them to forensically analyse an incident? We can get some hints by looking at the amount of review activity… Audit > Incident Review Audit and Audit > Suppression Audit are of course useful, but you can also do this sort of thing:
| `incident_review` | search status_default=false | timechart span=1day count by reviewer usenull=f
“Forensically analyze” …