Write your own search language

Splunk provides many power search commands — such as sort, fields, transactions — but even better, it allows you to expand things anyway you want, by writing your own search commands.

I’ll show you how to write your own search command.

Suppose you want to make a new “shape” command in python that returns the shape of an event — tall, short, thin, wide, etc.  There are just three simple steps:

  • Step 1) Tell splunk about this external command in  commands.conf…
[shape]
filename = shape.py
  • Step 2) Authorize users to run this command in authorize.conf…
[capability::run_script_shape]
[role_User]
run_script_shape = enabled
  • Step 3) Write the code!  Here is shape.py…
   import splunk.Intersplunk 

   def getShape(text):
        description = []
        linecount = text.count(”\n”) + 1
        if linecount > 10:
            description.append(”tall”)
        elif linecount > 1:
            description.append(”short”)
        avglinelen = len(text) / linecount
        if avglinelen > 500:
            description.append(”very_wide”)
        elif avglinelen > 200:
            description.append(”wide”)
        elif avglinelen < 80:
            description.append(”thin”)
        if text.find(”\n “) >= 0 or text.find(”\n\t”) >= 0:
            description.append(”indented”)
        if len(description) == 0:
            return “normal”
        return “_”.join(description)            

   # get the previous search results
   results,unused1,unused2 = splunk.Intersplunk.getOrganizedResults()
   # for each results, add a ’shape’ attribute, calculated from the raw event text
   for result in results:
        result["shape"] = getShape(result["_raw"])
   # output results
   splunk.Intersplunk.outputResults(results)

It works!  Show me the top shapes among events with more than one line…

$ splunk search "linecount>1 | shape | top shape"
shape                count  percent
-------------------  -----  ---------
tall_indented           43  43.000000
short_indented          29  29.000000
tall_thin_indented      15  15.000000
short_thin_indented     10  10.000000
short_thin               3   3.000000

Just to review, here are the files we made…

    apps/example/bin/shape.py
    apps/example/default/authorize.conf
    apps/example/default/commands.conf

Now go out there and make cool extensions to Splunk!

2 Responses to “Write your own search language”

  1. Daniel Says:

    Thanks for the example David.

    Any tips on how to test or debug custom search commands like this?

    I’m attempting to re-implement the “iplocation” command using the python geo-ip libraries and I’m not having great success. As a noob pythonista, it’d be handy to people able to see exactly what the my code is (not) doing.

  2. david Says:

    You can just call your search operator from the commandline, and give it what it wants for standard input.
    For the example above:

    1) type “python shape.py”

    2) then enter a blank line (carriage return) to tell splunk you’re about to enter the input results.

    3) then enter your sample input data, as a CSV, for example:

    _raw,source
    blah blah raw1, source1
    blah blah raw2, source2

    4) then enter control-d to end standard input, and your program should run on that sample data we just gave it.

Leave a Reply