Splunk Add-on > Where’s That Command – Converting a Field’s Hexadecimal Value to Binary

When looking through Splunk’s Search Reference Manual, there are a ton of search commands with their syntax, descriptions, and examples.  After all, if Splunk is the platform for machine data, there needs to be an extensive list of commands, functions, and references that guide Splunkers through the Search Processing Language (SPL).  But one would think that we had everything covered, right?  Well, almost….

I have a couple of great customers from the Houston, Texas area to thank for this.  Gabe and Andrew (you know who you are) are not only strong Splunkers, but frequent the Splunk Houston User Group (SHUG) meetings and are always looking for ways to expand their use of Splunk as well as get others just as passionate and excited about it as they are!  In two separate instances they brought me a simple question – Where’s that command that converts my hexadecimal values in this field to a binary number?

As I started digging into the Search Reference Manual and across our www.splunk.com website, I quickly found what many were already finding or found at answers.splunk.com – there is not a command that does this!  DOH!  Various people had ideas of building searches that included eval functions, even using the replace command (something I blogged about before here), but ultimately, no SPL-based command.  While it’s cool to have massive, multi-line search strings in your Splunk search bar, its not very efficient or a good use of time as compared with just doing a single command type call.

The first time I attempted to help with this it was an Energy-based use case that had some IT Security use case to it.  The second time I worked on this it was with a retail/point-of-sale analytics use case.  Regardless of the use case, what I quickly realized is that we needed something to make converting the hexadecimal values in fields to binary as simple as just flipping a switch…. or installing a Splunk Add-on.

Enter the Splunk Add-on – Hexadecimal to Binary Add-on (Hex2Binary Add-on)!

This is a fairly simple add-on which leverages the power of Splunk’s search macros.  You download the add-on and then use the “Manage Apps” to install the app from a file or use the new feature in Splunk 6.3.x to Browse More Apps to find and download the add-on:

Screen Shot 2016-02-11 at 2.44.47 PM

 

Once installed, the add-on is set to Global Sharing Permission which means any of your apps in Splunk should be able to leverage it.

Screen Shot 2016-02-11 at 2.49.53 PM

 

For documentation, please refer to the README.txt file in the “…etc/apps/SA_Hex2Binary/” directory:

 

Screen Shot 2016-02-11 at 3.13.10 PM

 

To use the “hex2binary()” macro, you use the SPL call format for Splunk macros but it requires you to pass the field which contains the hexadecimal values you wish to convert to binary.  As a simple test (since I was not able to use any of my Splunk customers’ data) I will create a field and give it one hexadecimal value:

* | eval hex_num=”BC55″

 

Screen Shot 2016-02-11 at 3.19.23 PM

 

Now that we have a field with a hexadecimal value, I can pass that field to the “hex2binary()” Splunk macro, where the binary conversion is placed into a field named “binary”:

* | eval hex_num=”BC55″ | `hex2binary(hex_num)`

 

Screen Shot 2016-02-11 at 3.22.57 PM

 

That is a LOT easier than having to write eval and loop statements into your search!

Enjoy the new add-on and should there be any questions or requests for enhancement/upgrades, please let me know!

Happy Splunking!

PD2

Hi Paul

Thanks for the add-on. However It doesn’t appear to work with all hex strings e.g. x’BC05′. I think the macro needs an additional replace eval statement before the current first replace eval statement to handle a nibble of 0.

eval binary = upper($hex$)
| eval binary = replace(binary,”0″,”0000″)
| eval binary = replace(binary,”1″,”0001″)
| …

Regards

Dave

Dave Brooking
February 12, 2016

My original goal was to convert hex to ascii text. Is there a function that converts your binary to text?

My initial solution to this problem was using the urldecode function like this:
https://answers.splunk.com/answers/151846/how-to-convert-hex-to-ascii-in-splunk.html

I created a macro based off of that until Splunk can bring us this feature:
hex2ascii(2)

eval hex_url=$fieldname_hex$ | rex mode=sed field=hex_url “s/([0-9A-Fa-f]{2})/%\1/g” | eval $fieldname_ascii$ =urldecode(substr(hex_url,3))

Also, I am interested in learning the use cases where a hex to binary conversion would be useful.

Andrew Landen
February 12, 2016

Hi Dave,

Thank you for your help – I totally missed that and for whatever reason in my testing I did not test a “0”. Good catch! I’ve actually made the changes to the add-on’s macro and have updated this to version 1.1! Feel free to download the update and bang on it.

Thank you again!
PD2

Paul Daigle
February 12, 2016

Hey Andrew!

I knew there was something different in your particular use case….but hey, it still had to do with hexadecimal conversions, right? :-)

You ought to take what I did here for the Hex2Binary and use it with the macro you created to create another add-on for that use case.

As for the use case where hex to binary conversion is useful, I’ll let you and Gabe have that conversation, if he’s willing to discuss it. :-)

Thank you!
PD2

Paul Daigle
February 12, 2016