Recently I had a request internally for how to access the Export endpoint from Splunk from a node.js application. The Export endpoint is useful for exporting large amounts of data efficiently out of Splunk as it will stream the results directly rather than requiring you to continually poll for more results. It turns out we don’t support the Export endpoint currently in our JS SDK, but it is very easy do access it yourself using Mikael’s super simple request module.
A picture (or a snippet in this case) tells a thousand words. Below you can see how to export Splunk’s internal index. Once you start it up it will instantly start streaming. Make sure you have enough disk space, or …
Splunk Command> Cluster
Being a Splunk sales engineer is incredible. I get to talk to customers about their use cases, ‘Splunk’ their data, and together discover the insight Splunk provides them. Initial demos typically start with the search bar, looking for keywords in their data. Usually doesn’t take long before the “Ah Hah!” moment comes – either by using Splunk’s intuitive GUI to interact with extracted fields of interest or employing a very small subset of the 130+ search commands with in the search bar to gain operation intelligence not readily seen before. At a recent customer visit I employed the Splunk on Splunk (S.o.S.) App, explored some of the underlying searches and noticed the cluster command, which I never used before. …
Updating the iplocation db
When Splunk added the new version of the iplocation command in v6.0, it added the ability to add location info without the need for internet concenttivity. We did this by shipping a custom version of the MaxMind DB in the 6.0.x release. However, because we used a Splunk specific version of the DB, you still had to wait for a new version of Splunk to get a new copy of the DB.
In 6.1 we added support for using the native MaxMind DB (.mmdb), allowing you to update the DB yourself at anytime! It looks like some of you have already figured this out (Go George go!), but I figured I would add some additional info about this …
Search Command> diff
What’s the grooviest Splunk search command goin’ round? It’s diff man, can you dig it?
That’s right, diff. What other command is based on a *nix file comparison utility that’s been around since the early 70’s?
Splunk’s diff operates just like good ol’ diff does on a *nix platform – it compares two inputs and tells you what the differences are, in a very distinct format. But where *nix diff normally compares two files, Splunk’s diff compares the content of two events.
We can use diff to compare one field in an event to that same field in another event, or we can go for broke and have diff compare “_raw” – or the content of the entire event …
Microsoft Patch Tuesday! Are your servers patched?
Enabling Real-Time Backfill
The funny thing about Splunk is how it just doesn’t stop surprising you. Even after years of using it, you still get surprises. Okay, I must confess I haven’t used Splunk for years, but you get the idea.
Last week, I was in the land of Kimchis in a -5 degrees Celsius (23 degrees Fahrenheit) room, wearing no gloves and a thin veil as my underwear. It was brutal as usual, especially when you are not in your own country. I was thumbing thru the Splunk User’s manual trying to look for some answers. Out of the corner of my eye, I saw a few paragraphs that were new to me. In fact, there were so new that I believed …
The effective security buyer
Sometimes I’m glad I keep around the mounds of free trade publications that tend to pile up on my desk. About once a month I start going through the stack but I never quite finish. I got through placing 4 or 5 publications into the recycling bin before I read How to be an effective security buyer by Andreas M. Antonopoulos in the May 2011 on-line edition of Network World. This article (more than others I’ve read) takes a basic and practical approach to the hard buying decisions (and trade-offs) we all have to make. The buying decision criteria is crisp and fresh and discourages simply relying on vendors that would “draw you into a single-vendor closed integration package” of …
Maintaining State of the Union
Well, it’s almost that time of year again already – the State of the Union address is scheduled for January 25th, 2011.
- Things are getting better :]
- There are still many challenges to overcome :[
- Inspirational story 1, with subject of said story in attendance to the left of Mrs. President ;_;
- Inspirational story 2, with subject of said story in attendance to the right of Mrs. President ;_;
- Wrap it up, B(arack) [comedycentral.com] :&
However, I would actually like to discuss a different kind of “state” – one that is more directly related to Splunk’s built-in capabilities (though I haven’t given up on my ‘Anti-unemployment’ or ‘Budget …
Astronomy and Summary Indexing
I had the pleasure last week of viewing Saturn’s rings at Rutgers University’s observatory. It was my first time actually seeing the rings through a professional telescope and the planet does look like what we often see in text book pictures. After the viewing, I started thinking that astronomy records a lot of data that needs to be indexed for search and aggregated for reports. I asked the professor conducting the tour if he had any logs for astrometry data and he took out his paper notebook to show it to me. Obviously, in Splunk terms, that was not what I was asking to see.
In seriousness, the professor told me that optical telescopes, radio telescopes, and spectrometers can generate …
Quick Splunk Reference for SQL Users
If you are familiar with SQL and think in SQL, this quick comparison might be helpful for you to dive into the Splunk search language. Splunk is not a database, in the normative sense, but there are enough similar concepts between the Splunk and the database worlds that this quickstart makes sense.