Write your own search language
| Topics: | Homepage, dev, hacks |
|---|---|
| Tags: | command, dev, developer, language, operator, python, search |
| Share: |
Splunk provides many power search commands — such as sort, fields, transactions — but even better, it allows you to expand things anyway you want, by writing your own search commands.
I’ll show you how to write your own search command.
Suppose you want to make a new “shape” command in python that returns the shape of an event — tall, short, thin, wide, etc. There are just three simple steps:
- Step 1) Tell splunk about this external command in commands.conf…
[shape] filename = shape.py
- Step 2) Authorize users to run this command in authorize.conf…
[capability::run_script_shape] [role_User] run_script_shape = enabled
- Step 3) Write the code! Here is shape.py…
import splunk.Intersplunk def getShape(text): description = [] linecount = text.count(”\n”) + 1 if linecount > 10: description.append(”tall”) elif linecount > 1: description.append(”short”) avglinelen = len(text) / linecount if avglinelen > 500: description.append(”very_wide”) elif avglinelen > 200: description.append(”wide”) elif avglinelen < 80: description.append(”thin”) if text.find(”\n “) >= 0 or text.find(”\n\t”) >= 0: description.append(”indented”) if len(description) == 0: return “normal” return “_”.join(description) # get the previous search results results,unused1,unused2 = splunk.Intersplunk.getOrganizedResults() # for each results, add a ’shape’ attribute, calculated from the raw event text for result in results: result["shape"] = getShape(result["_raw"]) # output results splunk.Intersplunk.outputResults(results)
It works! Show me the top shapes among events with more than one line…
$ splunk search "linecount>1 | shape | top shape" shape count percent ------------------- ----- --------- tall_indented 43 43.000000 short_indented 29 29.000000 tall_thin_indented 15 15.000000 short_thin_indented 10 10.000000 short_thin 3 3.000000
Just to review, here are the files we made…
apps/example/bin/shape.py apps/example/default/authorize.conf apps/example/default/commands.conf
Now go out there and make cool extensions to Splunk!

September 30th, 2008 at 11:52 pm
Thanks for the example David.
Any tips on how to test or debug custom search commands like this?
I’m attempting to re-implement the “iplocation” command using the python geo-ip libraries and I’m not having great success. As a noob pythonista, it’d be handy to people able to see exactly what the my code is (not) doing.
October 1st, 2008 at 7:30 pm
You can just call your search operator from the commandline, and give it what it wants for standard input.
For the example above:
1) type “python shape.py”
2) then enter a blank line (carriage return) to tell splunk you’re about to enter the input results.
3) then enter your sample input data, as a CSV, for example:
_raw,source
blah blah raw1, source1
blah blah raw2, source2
4) then enter control-d to end standard input, and your program should run on that sample data we just gave it.