Storing encrypted credentials

Splunk 4.2 was released today and your new resolution:

Build the greatest Splunk app that gathers data from all different source, some that are public and others that require credentials, index them in Splunk and then do some cool things with it.

This blog post will only be concerned with one small, but important aspect of your great app: how to securely store user credentials yet be able to safely access them in clear text when needed. I will split up the post into four sections: get credentials from the user,  access them from your script, where are the credentials stored and security implications.

Get and securely store user credentials

The best time to get user credentials for you app would be during app setup.  If your scripted input needs credentials to work properly you should ship it as disabled and then during app setup ask the user to supply the credentials and activate the input.  So, as you probably already know, Splunk provides a mechanism for asking the user to provide the necessary information to setup an app –  this utility in Splunk lingo is known as setup.xml, for more info see Configure a setup screen. In your setup screen you would want the user to provide you with a new set of credentials so that section of your setup.xml should look like:

    <block title="Add new credentials" endpoint="storage/passwords" entity="_new">
            <input field="name">
            <input field="password">
            <!-- feel free to comment realm out if your app is not using it -->
            <input field="realm">

The above block will gather: username, password and (optionally) realm and will create a new REST entity under the storage/passwords endpoint in your app. For those who don’t speak REST that basically means the credentials will be stored by Splunk somewhere, read further down to see exactly where they’re written.

Access credentials from your script

This section, for simplicity, assumes that your scripted input is written in python. To access and use the credentials requires that you:

(1) tell Splunk to pass the script an auth token that the script can use to access Splunk’s REST endpoints – in this case we just want to access the passwords endpoint.

passAuth   = splunk-system-user

(2) access and use the credentials that the user provided
import splunk.entity as entity
# access the credentials in /servicesNS/nobody/<YourApp>/storage/passwords
def getCredentials(sessionKey):
   myapp = 'name-of-your-app-here'
      # list all credentials
      entities = entity.getEntities(['admin', 'passwords'], namespace=myapp, 
                                    owner='nobody', sessionKey=sessionKey) 
   except Exception, e:
      raise Exception("Could not get %s credentials from splunk. Error: %s" 
                      % (myapp, str(e)))

   # return first set of credentials
   for i, c in entities.items(): 
        return c['username'], c['clear_password']

   raise Exception("No credentials have been found")  

def main():
        # read session key sent from splunkd
        sessionKey = sys.stdin.readline().strip()

        if len(sessionKey) == 0:
           sys.stderr.write("Did not receive a session key from splunkd. " + 
                            "Please enable passAuth in inputs.conf for this " +

        # now get twitter credentials - might exit if no creds are available 
        username, password = getCredentials(sessionKey)

        # use the credentials to access the data source 

… and that should be it! To recap we asked the user for credentials, stored them encrypted somewhere and used them in your script. If you don’t care where the credentials are stored and how secure they are feel free to skip the rest of this post and go finish your great app!!

Where are the credentials stored?

The encrypted credentials are stored in: $SPLUNK_HOME/etc/apps//local/app.conf in stanzas that look like this:

password = $1$<encrypted-password>

Please read $SPLUNK_HOME/etc/system/README/app.conf.spec for more info on the format.

Security implications

The passwords are encrypted and stored – note that we don’t store password hashes because we need to be able to decrypt it and provide you with the clear text password for the script to use. Since the encrypted data and the encryption key are stored on the same machine, cryptographically this is equivalent to obfuscation. While some people might argue that this is very weak encryption, it is the best we can do with storing the encrypted data and encryption key on the same machine and it is definitely better than clear text passwords :) To improve on the security we also:

– restrict access on storage/passwords endpoint to only system/admin users (users that have admin_all_objects capability)
– audit all accesses on the endpoint (run the following search to see the relevant audit entries: index=_audit action=*password)

Have fun building your app !!

This is killer! GO ALBANIA!!!1

One note: if you use entity.getEntity() to retrieve a password and then want to change the password using entity.setEntity(), unset the ‘clear_password’, ‘encr_password’, and ‘name’ members from the entity before sending to setEntity().

April 15, 2011

Just to add , to get the session token passed to a Scripted Input , you need to set the “passAuth” property in the inputs.conf stanza.

Furthermore , if you are using Modular Inputs, the session token is passed in the input xml when the modular input is executed by SplunkD.

Damien Dallimore
February 19, 2013

I’m trying to implement this feature in one of my apps, but splunkd is throwing an exception as shown below.

” Exception: Could not get TA-Sample credentials from splunk. Error: [HTTP 500] Splunkd internal error; [{‘type’: ‘ERROR’, ‘text’: “In handler ‘passwords’: Cannot base64 decode encrypted password”, ‘code’: None}]

Any inputs on this will be really helpful.


February 19, 2013

Kiran, can you please move your question on What are the contents of apps.conf for your app? When is this exception thrown – ie what request are you making?

February 20, 2013


Here is the code and app.conf


is_visible = 0
label = TA-accuvant


check_for_updates = 1

is_configured = 1

password = $1$XJHN09pBlRtp

Handler Python function

def getCredentials(session):

myapp = ‘TA-Accuvant’
# list all credentials

entities = entity.getEntities([‘admin’, ‘passwords’], namespace=myapp,
owner=’nobody’, sessionKey=session)
except Exception, e:
raise Exception(“Could not get %s credentials from splunk. Error: %s”
% (myapp, str(e)))

# return first set of credentials
for i, c in entities.items():
return c[‘username’], c[‘password’]

raise Exception(“No credentials have been found”)

February 25, 2013

Can you post a snipet that shows how to use setEntity to set the password of a particular username in local/app.conf


Bob Light
October 19, 2015

One Trackback

  1. […] for an alert action script in Splunk Cloud and wanted to share the details. Ledion Bitincka wrote a great article about storing encrypted credentials using the storage/passwords REST endpoint and accessing them in scripted inputs. This tactic is […]