Last week Splunk sponsored ComputerWorld’s Infrastructure World conference along with HP and IBM. I needed to come up with a talk and I wanted to do something new.

I’ve been thinking about how to describe the challenges we have managing all this changing technology and innovation. Note this is seriously a work in progress. I’m developing a theory that there are three fundamental drivers to data center chaos.

• expectations,
• complexity and
• accountability

Any new business or consumer technology can be quickly met with significant expectations if it becomes successful. Our dependence on everything from wireless email, online travel reservation systems and hosted software as a service dramatically increases the expectations these technologies will always be available, fast and do everything we want. Examples of failed expectation are everywhere. A few examples. On June, 20th United Airlines canceled 24 flights and delayed another 286 flights due to a “computer gremlin.” Research in Motion recently experienced yet another 24 hour email outage and more than 2.5M users were without service in North America. Salesforce.com, pioneers of Software as a Service (SAAS), a more reliable alternative to running it yourself continue to have outages as well.

This week I gave a talk at the SOAWorld Conference in New York. The focus was a discussion of recent SOA disasters and the challenges in managing large scale SOA architectures with examples from Citigroup, United Airlines, Research in Motion and Salesforce.com. We looked at the pluses and minuses of tools like business activity monitors, web session monitors, dependency mapping, change control and IT Search.

The audience was a mix of 75 software developers and IT architects. A good discussion followed about correlating large-scale data, anonymizing data sources and different models for mapping access controls to SOA message data.

Earlier this year