CVE-2015-7547: glibc: Detection with Splunk

This blog post will show you how to find if your systems are affected by the glibc vulnerability revealed on Feb 16th, 2016 by Google researchers. This vulnerability affects the glibc versions from 2.9 through 2.22. This is a critical bug because glibc is used across many mobile, virtual, cloud and high performance computing platforms and could lead to remote exploitation.

The Problem

According to the Google Blog post:

“The glibc DNS client side resolver is vulnerable to a stack-based buffer overflow when the getaddrinfo() library function is used. Software using this function may be exploited with attacker-controlled domain names, attacker-controlled DNS servers, or through a man-in-the-middle attack.

Google has found some mitigations that may help prevent exploitation if you are not able to immediately patch your instance of glibc. The vulnerability relies on an oversized (2048+ bytes) UDP or TCP response, which is followed by another response that will overwrite the stack. Our [Google] suggested mitigation is to limit the response (i.e., via DNSMasq or similar programs) sizes accepted by the DNS resolver locally as well as to ensure that DNS queries are sent only to DNS servers which limit the response size for UDP responses with the truncation bit set.”

A proof of concept exploit has been released and is publicly available. Discussion on technical details can be found at sourceware.

Impacted Systems

Glibc is vulnerable from version 2.9 until 2.22 (Get the latest updated version there: https://www.gnu.org/software/libc/libc.html ).

To find your glibc version, you can use the the ldd command:
$ ldd –version
ldd (Debian GLIBC 2.21-7) 2.21
$

Using Splunk to detect if you are vulnerable

Here are the steps we used to create a splunk app to detect this vulnerability. You can use a scripted input to check for the version of glibc.

  1. Create an app directory for the scripted input

$ mkdir etc/apps/glibc
$ mkdir etc/apps/glibc/bin
$ mkdir etc/apps/glibc/default

  1. Create an python script to collect the glibc version

$ cat >> etc/apps/glibc/bin/checklocal.py << EOF
> #!/usr/bin/python
>import subprocess
>import datetime
>import os
>def log(msg):
>   fp = open(os.path.join(os.environ[“SPLUNK_HOME”], “var”, “log”, “splunk”, “glibc-version.log”), “a”)
>   fp.write(“[%s] %s\n” % (datetime.datetime.now().isoformat(), msg))
>   fp.close()
>out = subprocess.check_output([“ldd”, “–version”])
>log(out)
> EOF

  1. Create an input to run the previous script every hour

$ cat >> etc/apps/glibc/default/inputs.conf << EOF
> [script://./bin/checklocal.py]
> disabled = false
> interval = 3600 # Every Hour.
> EOF

  1. This app directory should be installed on Splunk Forwarders running on the systems that may be vulnerable.

Once data is indexed by Splunk you can see the glibc version numbers your systems are running:

glibc

To update glibc to a patched version, use your system specific tools like yum on Redhat/CentOS systems or apt-get on Debian systems. For example on RedHat/CentOS you can issue the following command:

yum -y update glibc

Network-based Detection

You can also use some network based detections to check for this vulnerability. E.g. you can use your firewall logs to detect large DNS packets. In this example we use Palo Alto Networks firewall logs.

newsearch

A more efficient search for the above would use the criteria in the first pipe:

index=”pan_logs” dns bytes > 512 | sort – bytes | table src, dest, protocol, application, bytes

The above search looks for dns requests that are larger than 512 bytes for both udp and tcp.

Other Detection Methods

You can also use SNORT/Sourcefire with the proposed signature on BugTraq. Splunk has a Sourcefire Add-on

More mitigations can be found on arstechnica and sourceware.

Thanks,

Cedric Le Roux
Monzy Merza
Sebastien Tricaud

Excellent post, makes us realize just how powerful scripted inputs can be! I would like to point out that the DNS monitoring will not detect the actual vulnerability itself, but is detecting a potential exploit of a vulnerability. Something to pay attention to is a potential attack via DNS that is targeting a system we know is vulnerable, that should trigger an alert to react to above others. When you correlate vulnerability data with attack behavior, organizations can better identify threats they need to respond to immediately. This is where the power of Splunk comes in, the ability to correlate disparate data sources to identify real threats around these high impact vulnerabilities.

It should also be noted that these vulnerabilities will exist on systems that may not be able to run a forwarder or scripted inputs, such as network appliances etc. For this reason and also for larger organizations with a lot of assets, they should have a vulnerability management tool in place to regularly monitor their environments for vulnerabilities. We have a number of vulnerability management technology partners such as Qualys, Tripwire, Tenable and others who have apps in Splunkbase that integrate directly with Splunk. When CVE-2015-7547 was made public, all of them had content within 24 hours for detecting this vulnerability.

February 28, 2016