Blogs

This small, unofficial project integrates an open-source AIM (AOL Instant Messaging) Chatbot with Splunk 4, allowing ad hoc searching, running of prepared searches, and real-time search alerting via instant messaging.

What’s real-time searching? It’s new in Splunk 4.1, out shortly, and will allow users to search for “real-time” events, within seconds of them reaching Splunk. Most usefully, you can set up real-time searches and be IM’d with the matching events the second they show up. You could ask to be IM’d, for example, whenever someone logs into your system, whenever there’s an error, whenever someone logs in as root, etc.

Above is a screen capture of real-time alerts printing out for each time someone downloads Splunk!

Note: You can use this project with Splunk…

Unless you have had your head in the sand, SQL Injections have made a fierce comeback to the top of the threat vector charts this year. According to the WHID (Web Hacking Incidents Database), SQL injection is still king of the attack vectors, accounting for 19 percent of attacks, followed by authentication abuse (11 percent), content spoofing (10 percent), DDoS/brute force (10 percent), configuration/admin error (8 percent), cross-site scripting (8 percent), cross-site request forgery (5 percent), DNS highjacking (5 percent), and worms (3 percent).

Reflect on the recent increase in compliance legislation requiring businesses to provide dynamic data access to customers for banking, healthcare, or the influx simple purchases on the web, and the concern may be scarier for all of us.…

Here’s a document that can get you analyzing real data and making real charts, in about an hour or two…

Dive into Splunk

Feedback really, really appreciated.

Reports you could be making in about an hour!

From an operating-system perspective, splunk is a system of programs that work together to provide the utility that users experience. Each of these programs have their own memory use patterns, and having some idea of them is good for investigating memory exhaustion/performance problems, as well as resource planning.

The involved parties in the splunk memory picture are:

• the operating system
• splunkweb
• splunkd

Programs launched by splunkd:

• splunk-search
• python search processors
• splunk-optimize
• scripted inputs such as wmi, imap, regmon, admon, vmware, imap, or your own customized/created agents
• scripted alerts
• scripted index management scripts (warmtocold, coldtofrozen)
• scripted auth

Many of these (especially the scripts) are largely external to splunk, in that splunkd runs them as requested, but their resource consumption is up to third party…

Every once in a while, rarely, you may get a splunkd.log error that looks something like this:

12-07-2009 14:32:06.894 ERROR bucket - Failed to resurrect timezone ('
' delimited): '### SERIALIZED TIMEZONE FORMAT 1.0
C0
Y0 NW 47 4D 54
$'

This is splunk saying it can’t parse the timezone description it just got. This can be a problem when you’re in a distributed environment, and you’re asking for data to be bucketed (collected) into time-specific chunks. A typical example is when using timecharts.

The fix for this particular issue is called Splunk 4.0.7, but if you’re curious to know what timzeone it actually is, the digits of hex are the name, represented as ascii values.

A quick trip to python shows us a more familiar name:

jrodman@joshbook:~> python
Python 2.6.1…

The second presentation at the Boston Splunklive event on January 28th was an in-depth profile of a large-scale deployment in a financial services firm, anonymously described as “one of the world’s largest providers of financial services.” Paddy Griffin, Director of Technical Architecture, used his extensive history in the software industry to provide context to his firm’s plans with Splunk. Unlike other major IT projects at his firm, this Splunk-based initiative is being rolled out in record time, using an iterative approach, to show they can provide a continually enhanced log aggregation and search service as part of their “nimble infrastructure.”

Paddy started his presentation by unveiling the name of the overall initiative: LASSIE (yes, like the famous collie from TV). The…

120 users and prospects came together Thursday morning, January 28th, to attend the first Splunklive of 2010. Set at the Cambridge Marriott in Kendall Square, a major university and a major financial services firm presented on how they are using Splunk to better manage their IT infrastructures. Attendees came from the greater Boston area, Maine, Connecticut, and elsewhere in Massachusetts on a day when it was cold enough to walk across the Charles River.

The event was kicked off with a short overview of Splunk–a presentation followed by a product demo.

The first customer presentation was given by Jim Donn, Network Management Systems Engineer, and Tim Hartmann, Unix Systems Administrators. They requested that their university remain unnamed, so I’ll refer to them…

There are times when data within events contains sensitive information. This could be Social Security numbers, credit card numbers, date of birth, an employee’s salary information, etc. The data may be in the clear and when it gets sent to Splunk, it would be indexed. Any person in a role that has access to that data would be able to search on it. To prevent such things from happening, Splunk has an out of the box feature to mask sensitive data. For instance, a Social Security number may end up looking like xxx-xx-xxxx within a search. The administrator could either use Splunk’s built-in sed like syntax to replace sensitive strings or use a regular expression in a transforms.conf file to accomplish this.

This…

As a nod to our revolutionary approach to changing the way people monitor, report and interact with their IT data, the first SplunkLive of 2010 will kickoff in Boston, “The Cradle of Liberty.”

The event features three great customer speakers representing a large local university and one of the largest financial services firms in the US.

These IT pros are revolutionizing the way their organizations share, secure and troubleshoot the IT data critical to keep their operations running at top performance.

At the financial services firm, they truly are revolutionizing processes with Splunk. Typically a new product rollout could take up to two years, but because Splunk performed so well in the test environment, they’ve compressed the rollout to less than a 6 month…

Although Splunk Free shipped with 4.0.5, we’re getting a bunch of questions to support asking “where’s the Free?” Turns out actually turning Splunk Trial into Free-as-in-Beer Splunk may not be as obvious as we hoped.

When the trial expires, Splunk will automatically prompt you to get a trial extension, or convert to Free. However, if you’re ready to go to Splunk Free right away:

Go to Manager (from any app) -> License

Down in the text area of the license page, you’ll see text for “switch to a Free Splunk at any time” When you click that, you’ll go to the license switcher that will turn your Splunk into a Free Splunk.

click here for the free license

Note however, you must be running Splunk 4.0.5…