Splunk in Space: Splunking Satellite Data in the Cloud
This year a Team of Splunkers attended the ESA App Camp 2015 in lovely Frascati, Italy. The topic of this year’s challenge was:
“There are thousands of ways to enrich apps with data from space – what’s yours?”
The Splunk team featured Robert Fujara and Philipp Drieger alongside with camp participants Claire Crotty and Anthony Thomas. Together the team created a mobile web app that accessed a Splunk Cloud instance to analyze geolocation-based satellite data and inform users about different environmental indicators across Europe. Users can input their preferences in terms of living environment and based on different indicators they then receive recommendations on which city or region would suit them best.
The key data sources for this project included
- Satellite Data from ESA
- Real time analytics across multiple data sources
- Sentinel 3+5 type satellite data with 6 Layers (10.5 GB consisting of 15 million records)
- World Bank Data
- Splunk App for Twitter Data
- Eurostats Data for crime rates and country and city information across Europe
All this data was fed into Splunk Cloud, which subsequently presented the data through a mobile web app built utilizing Splunk JS SDK and Google Material Design.
To begin with the team used satellite data to create a matrix that split the landscape into buckets. After this, the team went through each bucket and built out information such as how green a certain area is and how it compares to others based on the satellite data. They also analyzed the data available for pollution levels in order to demonstrate how the natural environment in a region fared when compared with another. The factors considered ranged from green house gas emissions to other pollution levels like ozone, SO2 or NO2. The team was also able to integrate social media data to show which issues were currently being discussed online in particular areas.
To ensure that the tool could be used during other participants at the ESA Camp, the team created a nice interface based on the Splunk SDK and Google Material Design. Users can then adjust their preferences and based on this receive a list of the top cities that fit their requirements. The following images show the user interface on the web app and in the mobile apps.
The ESA App Camp 2015 was a great opportunity to explore data analytics with Splunk in new areas like satellite data. The team also showed how Splunk SDKs can be leveraged to build custom (web) applications. Now imagine what could be done with Splunk 6.3 choropleth maps for geoanalytics. Read more about the camp results here: ESA App Camp 2015 summary article.