Using Splunk as a data store for developers

A number of years ago, I wrote a blog entry called Everybody Splunk with the Splunk SDK, which succinctly encouraged developers to put data into Splunk for their applications and then search on the indexed data to avoid doing sequential search on unstructured text. Since it’s been a while and I don’t expect people to memorize the dissertations of ancient history (to paraphrase Bob Dylan), I’ve decided to write about the topic again, but this time in more detail with explanations on how to proceed.

Why Splunk as a Data Store?

Some may proclaim that there are many no-sql like data stores out there already, so why use Splunk for an application data store? The answers point to simplicity, …

» Continue reading

A sneak preview of TM Forum Big Data 2014

TMForumBigDataI’m lucky enough to be going to the TM Forum Big Data event in Amsterdam next week. It should be an interesting set of speakers and reference point for what telcos are doing with big data and also a great place to share best practice and practical experience.

I’m particularly interested to see the use cases for telco big data, looking at what data is being collected (from the network, handset, customer data) but also how that data is being used (customer experience, fraud, DevOps, marketing etc.)

We hear a lot about how big data allows better analysis, more informed decisions and identification of trends but in particular I’m keen to hear the presentation on how telcos are monetizing big …

» Continue reading

Five Must-See Splunk Demos at Cisco Live Milan

Splunk and Cisco are collaborating to unify security, networking, application and other data center silos with centralized, comprehensive operational intelligence.

Stop by Splunk booth E1/E2 at Cisco Live Milan January 27-31 to learn why Splunk is the ideal complement to your Cisco-based infrastructure plus check out a range of LIVE demos including …

Security

» Continue reading

Hunk Setup using Hortonworks Hadoop Sandbox

Hortonworks Sandbox is a personal, portable Hadoop environment that comes with a dozen interactive Hadoop examples. Recently Hortonworks and Splunk released a tutorial and video to install and connect Hunk with the Hortonworks Hadoop Sandbox version 1.3

This blog summarizes the configurations used as part of the Hunk setup.

Configurations for Hadoop Provider:

Key Value
Java Home /usr/jdk/jdk1.6.0_31
Hadoop Home /usr/lib/hadoop
Hadoop Version Hadoop version 1.x, (MR1)
Job Tracker sandbox:50300
File System hdfs://sandbox:8020
Splunk search recordreader com.splunk.mr.input.SimpleCSVRecordReader, com.splunk.mr.input.ValueAvroRecordReader

 

Configurations for Hadoop Virtual Indexes:

Key Value
Name hadoop_sports
Path to data in HDFS /user/hue/raanan/…
Whitelist \.csv$

 

 

For more Hunk details and examples go to the blog:

http://blogs.splunk.com/2013/11/08/hunk-intro-part-3/

Enjoy.

 …

» Continue reading

Splunk for Networking and SDN: In Action in Vegas!

Have you registered for Splunk Worldwide Users’ Conference .conf2013 in Vegas? If not, please hurry up and register here http://conf.splunk.com/. Time is running out!

Come to our session “Splunk for Networking and SDN” on October 1st, 1:45 PM to learn how Splunk can help you gain get deeper visibility into your existing networking infrastructure as well as your Software Defined Networking rollouts. We will discuss how to utilize Splunk to optimize your networking resources to cater to dynamically changing applications’ needs, reducing the number of monitoring solutions in your network, increasing security and of course, saving you money!

To see Splunk in action in the networking space, please stop by the booths and/or sessions of our technology partners Arista Networks …

» Continue reading

Happy SysAdmin day! I need to Splunk my brain – does your organization need to?

Hi. I’m having one of those weeks where I could do with Splunking my brain. Why? Because one thought keeps firing off another activity and adding to the unstructured list of things that I need to do. Essentially – it is working a bit like this:

What I really need, is it to work like this:

I’m sure we’ve all had times like this – lots of data coming at you that fits the mythical “three Vs”. There’s a high volume of data, it is moving quickly to give it velocity and there’s a lot of variety. What further adds to the need to Splunk my brain is the fact the data is at so many different …

» Continue reading

Big data and financial services – an EMEA perspective

I was lucky enough to attend the first day of the “Big Data in Financial Services” event in London a few days ago. I know some people might not think of that as lucky but I say it on the back of a surprisingly varied agenda, entertaining speakers and a lot of good debate and content on what big data means to FS companies and how they are using it.

The key point that I took away was that right now, FS companies are using big data today to focus on operational issues – risk, efficiency, compliance, security and making better decisions. However, there is a growing trend in FS companies looking at how big data is going …

» Continue reading

Splunk Joins Public-Private Partnership to Improve Cybersecurity

Last week Splunk joined several other companies at U.S. NIST’s signing ceremony symbolizing our participation and partnership in the National Cybersecurity Center of Excellence (NCCoE).

There’s no doubt that there is a critical need to protect private-sector intellectual property and other valuable business data from a growing number of cyber threats. This partnership illustrates our commitment to the spirit of collaboration while providing real-world cybersecurity capabilities that address business needs.

The NCCoE has three key goals:

  • Provide practical cybersecurity – Help people secure their data and digital infrastructure by equipping them with practical ways to implement cost-effective, repeatable and scalable cybersecurity solutions.
  • Increase rate of adoption – Enable companies rapidly adopt commercially available cybersecurity technologies by reducing their total cost
» Continue reading

The 2nd Annual APAC Partner Kick Off

Before we knew it, it is almost time for our 2nd annual APAC partner kick off that will fall on 19th till 21st March 2013 at magnificent Bali, Indonesia. As a preview to all the partners who will be attending this kick off with us, we have lined up a series of rock solid business and technical tracks that will definitely keep their time away from the beaches and bars. I’m sure the partners will gain tremendous values out from these three days.

There are close to 30 tracks that will be delivered, and topics range from global deployment considerations to Splunk modular inputs and SDK to even a rare chance to hear what our legal has to say about …

» Continue reading

Splunk Hadoop Connect 1.1 – Opening the door to MapR; now available on all Hadoop distributions

I am happy to announce that Splunk Hadoop Connect 1.1 is now available. This version of Hadoop Connect rounds out Splunk’s integration with the Hadoop distributions by becoming certified on MapR. Cloudera, Hortonworks, and Apache Hadoop distributions also have the ability to benefit from the power of Splunk.

Splunk Hadoop Connect provides bi-directional integration to easily and reliably move data between Splunk and Hadoop. It provides Hadoop users the ability to gain real-time analysis, visualization and role based access control for a stream of machine-generated data. It delivers three core capacities: Export data from Splunk to Hadoop, Explore Hadoop directories and Import data from Hadoop to Splunk.

The most significant new feature added to version 1.1 is the …

» Continue reading