Sunday, May 16, 2010

Beware small, individual decisions

Several of the NOLA Bloggers are drawing parallels between the ongoing BP oil geyser disaster and the catastrophic failure of floodwalls protecting New Orleans in 2005. I certainly agree that there may be similarities in these two different engineering debacles, but we have to be careful.

There is still a lot we don't know about what happened out in the Gulf of Mexico last month and why all the safeguards, standards, precautions, fail-safe systems, redundancies and the workers who lost their lives were not able to avoid or contain the dangerous conditions of mineral extraction. Until we do know--and that may be years--we need to keep an open mind to all the possibilities.

In contrast to the BP disaster, we now know a great deal about what happened and why before, during and after Hurricane Katrina. At different locations where constructed features did not perform properly we know there were specific reasons that those point failures occurred.

But the most significant and dangerous condition was not the result of a single point of failure or isolated bad engineering decision. The most devastating factor in my opinion was what is known as "The Tyranny of Incremental Decisions."

Simply put, during the long trek from project conception to operation, a long list of changes and compromises were made that individually didn't seem significant but which added up in a project doomed to failure. Some of those decisions may have been the result of budget pressures, or local preferences, or just ill-informed good intentions. Poor Richard advised that small strokes fell great oaks; The Tyranny of Incremental Decisions can be viewed in that manner.

As we learn more about the BP oil geyser disaster, I won't be surprised if the news media, politicians and many private citizens clamor for a single point of failure, a "smoking gun" that can be blamed. There may well be one. But I would not be surprised to learn that a series of decisions made by different people at different times had a large role to play in undermining the overall safety and reliability of the drilling operation.

(If you want to read the whole sordid tale about how The Tyranny of Incremental Decisions left New Orleans vulnerable, check out the Hurricane Protection Decision Chronology, or read the synopsis as reported in The New York Times here: Engineers Faulted on Hurricane System.)

(Editorial edits on 17 MAY 10)


Maitri said...

That's my concern, too, Tim. How well does "a culture of safety" scale within an organization or industry? How do small, poor decisions have to line up to create a tragedy like this one?

sussah said...

One thing we can say, however, is that the BP tragedy is a 100% man-made disaster. sp, n.o.

Sophmom said...

Back when I was in the employ of Mississippi River Land Airlines (not its real name), we had days of recurrent training every year, always including study of NTSB crash reports. It was always a combination of multiple failures that led to disaster. That said, my employer intentionally cultivated "a culture of safety" that I think we see much less of from corporations these days.

KC King said...

The tyrany of incremental decisions is certainly one of the root causes but what is the solution? I can say with absolute certainty that the solution in the aerospace domain is rigorous systems engineering. What about the water domain?

The IPET observed that the pre-Katrina flood protection works were a system in name only and that any future flood systems should be "good" systems. I would conjecture that the BP Deepwater Horizon catastrophe happened for the same reason - it was a system in name only!

Unfortunately, no one seeems to be asking that question including respected scientists such as Bob Bea. Every little snapshot we see on specific behaviors on the well indicate a violation of good systems practices from a number of perspectives.

The fact that we did not have the 8/29 commission to broadcast the IPET findings ensure that other domains would not see or learn the lessons we are still suffering.

As far as New Orleans flood protection is concerned we are still waiting for the controlling stakeholders to tell us how they have implemented the IPET's findings and recommendations.

See you at the RT5 where I hope there will an opportunity to address the unaddressed systems perspective.