Hunk Setup using Hortonworks Hadoop Sandbox
Hortonworks Sandbox is a personal, portable Hadoop environment that comes with a dozen interactive Hadoop examples. Recently Hortonworks and Splunk released a tutorial and video to install and connect Hunk with the Hortonworks Hadoop Sandbox
This blog summarizes the configurations used as part of the Hunk setup.
Configurations for Hadoop Provider:
|Hadoop Version||Hadoop version 1.x, (MR1)|
|Splunk search recordreader||com.splunk.mr.input.SimpleCSVRecordReader, com.splunk.mr.input.ValueAvroRecordReader|
Configurations for Hadoop Virtual Indexes:
|Path to data in HDFS||/user/hue/raanan/…|
For more Hunk details and examples go to the blog:
Splunk for Networking and SDN: In Action in Vegas!
Have you registered for Splunk Worldwide Users’ Conference .conf2013 in Vegas? If not, please hurry up and register here http://conf.splunk.com/. Time is running out!
Come to our session “Splunk for Networking and SDN” on October 1st, 1:45 PM to learn how Splunk can help you gain get deeper visibility into your existing networking infrastructure as well as your Software Defined Networking rollouts. We will discuss how to utilize Splunk to optimize your networking resources to cater to dynamically changing applications’ needs, reducing the number of monitoring solutions in your network, increasing security and of course, saving you money!
To see Splunk in action in the networking space, please stop by the booths and/or sessions of our technology partners Arista Networks …
Happy SysAdmin day! I need to Splunk my brain – does your organization need to?
Hi. I’m having one of those weeks where I could do with Splunking my brain. Why? Because one thought keeps firing off another activity and adding to the unstructured list of things that I need to do. Essentially – it is working a bit like this:
What I really need, is it to work like this:
I’m sure we’ve all had times like this – lots of data coming at you that fits the mythical “three Vs”. There’s a high volume of data, it is moving quickly to give it velocity and there’s a lot of variety. What further adds to the need to Splunk my brain is the fact the data is at so many different …
Big data and financial services – an EMEA perspective
I was lucky enough to attend the first day of the “Big Data in Financial Services” event in London a few days ago. I know some people might not think of that as lucky but I say it on the back of a surprisingly varied agenda, entertaining speakers and a lot of good debate and content on what big data means to FS companies and how they are using it.
The key point that I took away was that right now, FS companies are using big data today to focus on operational issues – risk, efficiency, compliance, security and making better decisions. However, there is a growing trend in FS companies looking at how big data is going …
Splunk Joins Public-Private Partnership to Improve Cybersecurity
Last week Splunk joined several other companies at U.S. NIST’s signing ceremony symbolizing our participation and partnership in the National Cybersecurity Center of Excellence (NCCoE).
There’s no doubt that there is a critical need to protect private-sector intellectual property and other valuable business data from a growing number of cyber threats. This partnership illustrates our commitment to the spirit of collaboration while providing real-world cybersecurity capabilities that address business needs.
The NCCoE has three key goals:
- Provide practical cybersecurity – Help people secure their data and digital infrastructure by equipping them with practical ways to implement cost-effective, repeatable and scalable cybersecurity solutions.
- Increase rate of adoption – Enable companies rapidly adopt commercially available cybersecurity technologies by reducing their total cost
The 2nd Annual APAC Partner Kick Off
Before we knew it, it is almost time for our 2nd annual APAC partner kick off that will fall on 19th till 21st March 2013 at magnificent Bali, Indonesia. As a preview to all the partners who will be attending this kick off with us, we have lined up a series of rock solid business and technical tracks that will definitely keep their time away from the beaches and bars. I’m sure the partners will gain tremendous values out from these three days.
There are close to 30 tracks that will be delivered, and topics range from global deployment considerations to Splunk modular inputs and SDK to even a rare chance to hear what our legal has to say about …
Splunk Hadoop Connect 1.1 – Opening the door to MapR; now available on all Hadoop distributions
I am happy to announce that Splunk Hadoop Connect 1.1 is now available. This version of Hadoop Connect rounds out Splunk’s integration with the Hadoop distributions by becoming certified on MapR. Cloudera, Hortonworks, and Apache Hadoop distributions also have the ability to benefit from the power of Splunk.
Splunk Hadoop Connect provides bi-directional integration to easily and reliably move data between Splunk and Hadoop. It provides Hadoop users the ability to gain real-time analysis, visualization and role based access control for a stream of machine-generated data. It delivers three core capacities: Export data from Splunk to Hadoop, Explore Hadoop directories and Import data from Hadoop to Splunk.
The most significant new feature added to version 1.1 is the …
Hadoop and Splunk Use cases
Customer Examples – Using both Splunk and Hadoop
The Splunk and Hadoop communities can benefit from each other’s strengths. Below are several examples of customers that use both environments.
|1 – Splunk then Hadoop||Splunk collects, visualizes, and analyzes the data and passes it to Hadoop for ETL and other batch processing|
|2 – Hadoop then Splunk||Hadoop Collects the Data, and passes the results to Splunk for Visualization|
|3 – Data flows in both directions||Splunk and Hadoop collect different artifacts and share the data that Hadoop needs for ETL or batch analytics and Splunk needs for real-time analysis and visualization|
|4 – Side-by-Side||Both Splunk and Hadoop are used by the organization, but are used|
You’re happier with fewer friends
Using the new Splunk Sentiment Analysis app I was able to correlate how positive tweets were, depending on how many people follow a twitter account. It’s a slight stretch, but essentially, are you happier with more friends?
index=twitter | sentiment twitter body | chart avg(sentiment) by actor.followersCount
It seems that people with smaller circles of friends are more positive. More friends equals more negativity, up until about 75 friends. Seems like a fairly good life lesson, but take it a grain of salt — spam twitter accounts may skew things.…
Simplifying Big Data Analytics
Most analytics and data projects have started thinking of investing in big data initiatives. With so much buzz about big data, organizations have started investing or are thinking of investing in Hadoop While it is great to stay on top of trends, it often ends up being another investment where the full benefit and potential is simply not realized. The learning curve is too steep and the time to implement too high. Current analytics resources lack the strong programming skills required to conduct even simple analysis tasks and activities using Hadoop. In this post, I would like to focus on providing a better understanding of what types of analysis are better suited for Hadoop vs. non-Hadoop technologies in order to simplify …