101 things the mainstream media doesn’t want you to know about PowerShell logging*


At .conf2016 Steve Brant and I presented on how to detect PowerShell maliciousness using Splunk [2]. The only problem is, if you didn’t attend the conference and only read the PowerPoint slides you might say something like “Your presentation is just big photos and SPL”. Which is true. Frankly, we like big fonts and we cannot lie. You other presenters may deny. That when a deck goes up with a big sans-serif font and a bright image in your eyes you get… distracted by where I am going with this paragraph. As such, we are going to create blog postings of our presentation for those of you who didn’t attend our talk in person. In this missive …

» Continue reading

Spotting the Adversary… with Splunk

Howdy Ya’ll. Eventually there is a Rubicon to cross in every Security professional’s life. With a satisfied sigh he’ll take a step back from the keyboard, wipe Dorito dust covered hands on khakis, take a long slug of Mountain Dew, and gaze proudly at his Splunk instance and utter the words “I’ve added all the data sources I can. The network is being ‘monitored’”. Then the smile will falter as his cyber demons claw their way up to the surface.  He’ll hear them scream out “but WHAT am I supposed to look for??”  He (and you) are not alone. Ever since time immemorial (or at least when I first began “practicing” the dark arts of cyber security) I would hear the question of “but what …

» Continue reading

Splunk 6.2 Feature Overview: Perfmon Delocalization

Last week, I covered the XML Event Logs – an awesome feature that will reduce your data ingest, increase the fidelity of the data that is stored and allow us to work with localized data. Today, I want to discuss another localization feature – or at least a delocalization feature – perfmon.

Prior to Splunk 6.2, Windows perfmon was always collected localized. If you wanted the % Processor Time counter, you had to specify the localized version of this. If you were running on a french version of Windows, you would have to specify object=Processeur and counter=”% Temps Processeur” in both your inputs.conf and searches. Given that there are over 30 different localized versions of Windows, this really meant that …

» Continue reading

Splunk 6.2 Feature Overview: XML Event Logs

We’ve been (rightly) criticized for a couple of things in recent years. Firstly, when you configure a Windows Event Log, it’s too big. This is because we combine the event log object with the message from the locale-specific DLL and that includes a bunch of common explanatory text. I don’t really need to know what a login really means (to the tune of 1K of data ingest) every time someone logs in, especially when these events are happening hundreds of times a minute. Secondly, our event log extractions are for US/English only. Got German Windows? Sorry – our extractions don’t work for that. Finally, we discard the additional data that is provided in the event log object. A primary example …

» Continue reading

Integrating Active Directory into Splunk with SA-ldapsearch

On Tuesday, I introduced one of the first presentations at .conf2014 – a major update to the SA-ldapsearch app. This new app has now launched and you can download it at http://apps.splunk.com/app/1151/. The app consists of four specific commands: ldapsearch, ldapfetch, ldapfilter and ldapgroup.

Improvements include:

  • We dropped the requirement for Java on your search head
  • We added support for Search Head Pooling
  • We added a GUI configuration page and connection testing
  • We provided full UTF-8 support

The ldapsearch command is a generating command and is used in a similar way to other generating commands like inputlookup. You run it like this:

| ldapsearch domain=SPL search="(objectClass=user)" attrs="sAMAccountName,cn"

We have added some new features in this release. Firstly, the output …

» Continue reading

Quick PowerShell Script to Start Splunk

Got another quick PowerShell post for you. I have a copy of Splunk running locally on my Windows 8.1 workstation. I don’t always leave it running, for obvious resource reasons, therefor I end up starting it and stopping it as needed. On Windows, there’s two ways to control the Splunk services:

  • CLI splunk.exe start|stop|restart commands
  • Windows native service control methods (and there’s a half-dozen ways to do that)

So, in PowerShell, you can just do this:

Get-Service splunk* | Start-Service

The only minor problem is that I keep forgetting to elevate my PowerShell shell, so I’ll get an error message, and then I have to open a new window, and then repeat the process.  That’s no way to automate, I said to myself, so I made this quick …

» Continue reading

Quick Tip: Upload Logs to Splunk from Windows PowerShell

I had a folder full of log files I wanted to index real quick in my local instance of Splunk. They won’t persist, so the right thing to do is to use the “oneshot” command (documented here). This can be done in the web UI, but I like doing stuff at the command line. I opened up PowerShell (elevated, as my Splunk instance runs as system) and tried this:

splunk add oneshot *.log

And this was the output:

In handler 'oneshotinput': unable to open file: path='C:\Users\Hal\temp\*.log' error='The filename, directory name, or volume label syntax is incorrect.'

It didn’t work! Ok, so my assumption was that Splunk would parse the wildcard and have at it. But no big deal, this is quick to …

» Continue reading

Install Splunk with PowerShell (2014 Edition)

One of our avid twitter followers asked how to reliably install the Splunk Universal Forwarder on a Windows host with PowerShell last week. I’ve posted about all the intricacies involved before but improvements in open-source tools for PowerShell have made it a whole lot easier. You can take a look at the original article, but follow along here instead. We’re going to walk through what’s involved.

Installing as a Local SYSTEM user is easy. Here is the recipe:

Invoke-Command –ComputerName S1,S2,S3 –ScriptBlock { `
New-PSDrive S –Root \\SPLUNK\Files -PSProvider FileSystem; `
Start-Process S:\splunkforwarder-6.1.1-207789-x64-release.msi `
    –Wait -Verbose –ArgumentList (`
        “AGREETOLICENSE=`”Yes`””, `
        “/Liwem!”, “C:\splunkinstall.log” ) `

Let’s recap what you need to do to install a Splunk Universal …

» Continue reading

Monitoring Windows Service State History

For Microsoft Windows Operating Systems, it is extremely important that certain services are running to maintain functionality.  For instance, if your Remote Procedure Call (RPC) Service is not running, your machine is most likely jacked up.  This article will show you how to monitor Windows Services and get a history of service state change history.

Getting Windows Service Data into Splunk

In order to monitor Windows Services, we have to get that data into Splunk. There are basically 2 built-in ways to do this:

Method 1 – use WinHostMon

Install a Splunk Universal Forwarder on your Windows host and use a WinHostMon stanza in your inputs.conf file. Here is what it looks like:

type = service
interval = 300
» Continue reading

Controlling 4662 Messages in the Windows Security Event Log

You’ve just installed the Splunk App for Windows Infrastructure, or its friend the Splunk App for Exchange. You’ve followed all the instructions, placed the Universal Forwarders on the domain controllers, and configured everything according to the documentation. Now your license is blowing up because you are getting too many EventCode=4662 in the Windows Security Event Log. How did this happen?

Security EventCode 4662 is an abused event code. It is used for directory access, like this:

An operation was performed on an object. 
Subject : 
    Account Name: EXCH2013$ 
    Account Domain: SPL 
    Logon ID: 0x177E5B394
    Object Server: DS 
    Object Type: domainDNS 
    Object Name: DC=spl,DC=com 
    Handle ID: 0x0 
    Operation Type: Object Access 
    Accesses: Control 
» Continue reading