ING Bank at Gartner Symposium. Delivering business value from operational insights.

ING_logo-1024x768Last week was EMEA’s Gartner Symposium and it was a pretty busy week. Thousands of CIOs, senior IT leaders and IT companies converged on a very windy Barcelona. We were lucky enough to have ING Bank speaking about how it uses Splunk to deliver business value from IT and ensure its customers are happy. ING Bank Slaski in Poland has over four million customers monitored by Splunk. ING’s IT goal is to make sure they are listening to the voice of the customer “to stay a step ahead in life and business” by:

  • Making it clear and easy to use ING’s banking services
  • Allowing customers access to those services anytime and anywhere
  • Empowering users to self serve and make use
» Continue reading

How to: Splunk Analytics for Hadoop on Amazon EMR.

Using Amazon EMR and Splunk Analytics for Hadoop to explore, analyze and visualize machine data

Machine data can take many forms and comes from a variety of sources; system logs, application logs, service and system metrics, sensors data etc. In this step-by-step guide, you will learn how to build a big data solution for fast, interactive analysis of data stored in Amazon S3 or Hadoop. This hands-on guide is useful for solution architects, data analysts and developers.

This guide will see you:

  1. Setup an EMR cluster
  2. Setup a Splunk Analytics for Hadoop node
  3. Connect to data in your S3 buckets
  4. Explore, visualize and report on your data

You will need:

  1. An Amazon EMR Cluster
  2. A Splunk Analytics for Hadoop Instance
  3. Amazon
» Continue reading

Caching Hadoop Data with Splunk and Hunk

Update 9/27/16: As of Sept. 27, 2016, Hunk functionality has been incorporated into the Splunk Analytics for Hadoop Add-On and Splunk Enterprise versions 6.5 and later.

Although Hadoop is good at processing a large amount of data, it is not the fastest platform. Below are a list of options that Splunk and Hunk can offer to speed up the retrieval of results and lower the processing overhead of Hadoop.

Each option has its own advantages:

Screen Shot 2015-05-05 at 11.54.16 AM

 

1) Hunk Report Acceleration

This option caches the results in HDFS and keeps it fresh and current.  By default, Hunk will check for new Hadoop data every 10 minutes.

Details =

http://docs.splunk.com/Documentation/Hunk/latest/Hunk/Workwithreportacceleration

 

2) Hunk Scheduled Searches

This option caches the results on the …

» Continue reading

Christmas 2020. Will big data and IOT change things for Father Christmas? Part II

ChristmasTreeBinary3

 

 

 

In part 1 we discussed how Father Christmas is planning to use sensor data for the Internet of Toys.

 

In part 2 we’re going to discuss how he is going to use very large data sets to build out his Christmas 2020 technology strategy.

 

 

 

 

 

Big Data & Analytics

There’s a lot of information that goes into making Christmas a success. This data includes:

  • Social media sentiment about good or naughty children
  • Christmas present lists from children (both digitized scanned letters and increasingly electronic present lists)
  • Data from toys and manufacturing equipment to spot patterns in quality control
  • 500 years of Christmas Eve delivery data to help optimize sleigh route planning
  • Reindeer biometric information to ensure optimum
» Continue reading

From big data to a 360 degree customer view with Hunk and Hortonworks

Update 9/27/16: As of Sept. 27, 2016, Hunk functionality has been incorporated into the Splunk Analytics for Hadoop Add-On and Splunk Enterprise versions 6.5 and later.

Age of the customerYou can’t really escape the fact that we’re in the age of the customer. From CRM to the “long tail” to multi-channel to social media brand sentiment to Net Promoter Scores – it is all about customer experience. Big Data has an important part to play – no great revelation there but how do you actually do it? There are an awful lot of questions that come up when it comes to Big Data and customer view;

What should my architecture be? How do I put together the right data strategy for the short …

» Continue reading

Get Value Out of Your Data in Hadoop, Starting Today

Update 9/27/16: As of Sept. 27, 2016, Hunk functionality has been incorporated into the Splunk Analytics for Hadoop Add-On and Splunk Enterprise versions 6.5 and later.

For years we’ve been working with thousands of companies using Splunk for big data solutions that range from security to business analytics and everything in between. The best part is our customers often discover exciting ways to use Splunk and teach us what the product can really do. As you can imagine, all of the customer conversations, product implementations and ROI stories have given Splunk a treasure trove of experience with big data and big data solutions.

So when our customers let us know that getting large amounts of data into Hadoop is straightforward, …

» Continue reading

Hunk Setup using Hortonworks Hadoop Sandbox

Update 9/27/16: As of Sept. 27, 2016, Hunk functionality has been incorporated into the Splunk Analytics for Hadoop Add-On and Splunk Enterprise versions 6.5 and later.

Hortonworks Sandbox is a personal, portable Hadoop environment that comes with a dozen interactive Hadoop examples. Recently Hortonworks and Splunk released a tutorial and video to install and connect Hunk with the Hortonworks Hadoop Sandbox version 1.3

This blog summarizes the configurations used as part of the Hunk setup.

Configurations for Hadoop Provider:

Key Value
Java Home /usr/jdk/jdk1.6.0_31
Hadoop Home /usr/lib/hadoop
Hadoop Version Hadoop version 1.x, (MR1)
Job Tracker sandbox:50300
File System hdfs://sandbox:8020
Splunk search recordreader com.splunk.mr.input.SimpleCSVRecordReader, com.splunk.mr.input.ValueAvroRecordReader

 

Configurations for Hadoop Virtual Indexes:

Key Value
Name hadoop_sports
Path to data in HDFS /user/hue/raanan/…
Whitelist
» Continue reading

Big data and financial services – an EMEA perspective

I was lucky enough to attend the first day of the “Big Data in Financial Services” event in London a few days ago. I know some people might not think of that as lucky but I say it on the back of a surprisingly varied agenda, entertaining speakers and a lot of good debate and content on what big data means to FS companies and how they are using it.

The key point that I took away was that right now, FS companies are using big data today to focus on operational issues – risk, efficiency, compliance, security and making better decisions. However, there is a growing trend in FS companies looking at how big data is going …

» Continue reading

Further Simplifying Big Data Analytics

In the past we’ve talked about simplifying big data analytics and the 80:20 rule for data analysis. Most organizations spend 80% of analytics efforts running and optimizing the business and 20% on advanced analytics, which includes advanced data mining, algorithm development and advanced predictive modeling.

Hadoop has seen very good adoption for big data analytics, specifically batch analytics for large datasets, and many organizations have initiatives to use it for advanced analytics and optimizing the business. Unfortunately, those organizations are struggling to derive value from their Hadoop implementations. They’re finding that analysis takes too long and requires specialized talent. Another issue is that getting data into Hadoop is difficult, getting meaningful analysis even more challenging.

In the past few months, …

» Continue reading

Splunk Hadoop Connect 1.1 – Opening the door to MapR; now available on all Hadoop distributions

I am happy to announce that Splunk Hadoop Connect 1.1 is now available. This version of Hadoop Connect rounds out Splunk’s integration with the Hadoop distributions by becoming certified on MapR. Cloudera, Hortonworks, and Apache Hadoop distributions also have the ability to benefit from the power of Splunk.

Splunk Hadoop Connect provides bi-directional integration to easily and reliably move data between Splunk and Hadoop. It provides Hadoop users the ability to gain real-time analysis, visualization and role based access control for a stream of machine-generated data. It delivers three core capacities: Export data from Splunk to Hadoop, Explore Hadoop directories and Import data from Hadoop to Splunk.

The most significant new feature added to version 1.1 is the …

» Continue reading