Water ALERT! by TCS, Thailand

My life seems to be revolving around disasters. From the earthquakes in Tokyo, to the hurricanes in Taipei, to the floods in Thailand and the once in a-hundred-year fog in Beijing. Been there, done that and have survived. Strangely enough, most of my colleagues seem to be reluctant to travel with me nowadays, for fear of getting caught in a calamity. Superstitious, they must be.

It has been tough going for the past few weeks in Thailand with massive flooding occuring in the northen and central regions of the country. The situation has now spread to the capital, Bangkok. During this ongoing crisis, timely information about the water levels of rivers and canals are critical and essential to the people …

» Continue reading

Unstructured machine data: a strategic business asset

Today, Splunk announced it’s receipt of a patent for it’s ability to, “…organize and understand machine data through use of a “machine data web.” A Splunk‐created machine data web can be searched, browsed, navigated, analyzed and visualized enabling IT professionals and businesses to solve a wide range of mission-­‐critical problems, all without the inherent limitations of traditional approaches.” How is this accomplished?  Unstructured machine data is given structure through time-based indexing so that analytics can be applied to the data to gain insight and understanding. I think of this as the key to understanding the secret life of machines and a reflection of human activities.

The effect of the “machine data web” on business will be felt in many ways. …

» Continue reading

Splunk and Astronomy, Part 2

I recently attended a series of fascinating short lectures, given in layman’s terms for the average audience, conducted by Astronomer Andy Green from Stardome (based in the UK). The topics that were covered included lunar landings, star formation, and planetary impacts. It is the last topic that I would like to discuss here. Andy presented a history of terrestrial and extra-terrestrial impacts for things like when an asteroid or a comet strikes a heavenly body. The most interesting heavenly body is, of course, the Earth itself. Today, there are a handful of organizations that track Near Earth Objects (NEO) and the log files that they collect can be indexed by Splunk. In the past I have already mentioned one use

» Continue reading

Universally Indexing Business Data

By the title of this entry, you may be thinking that there is some new capability within Splunk to index other types of data. That’s not the intention. From its roots, Splunk was used to index and search on IT data. It still is. However, because of the flexible nature of the software to index any type of time series text data, customers using Splunk do not restrict it to indexing only IT data. From the beginning Splunk was designed to universally index data from a variety of sources as long as the data was eventually ASCII text in representation.

Due to this inherent capability, Splunk can index data that is not necessarily meant for consumption by IT staff and …

» Continue reading

Indexing Events Delivered by Multicast

Although the title of this entry says indexing events delivered by multicast, the first thing I need to point out is not to do it. If you are indexing log type events, it is not a good idea to multicast this data to all machines on the LAN just to have one Splunk indexer that is listening for it to index it. Since most of the machines on the LAN won’t be interested in this data, It would be a waste of network resources, not to mention potentially unreliable.

Having said that, there may be cases where indexing events, such as control data, that are delivered via multicast may be useful. For instance, application servers in a cluster often are …

» Continue reading