How complex systems fail (a classic from Richard Cook)

Another classic from Richard Cook, where he covers 18 principles on how complex systems fail.

The paper is brief, freely accessible and an easy read.

I’ve extracted the 18 principles with a small text description in the below images.

Some random highlights for me are:

Catastrophic failures tend to involve “small, apparently innocuous failures” where each small failure is necessary but only in combination with other factors to permit failure
That is, many “causal” factors may be necessary but only jointly sufficient
Complex systems tend to always run as “broken systems”. That is, there’s always the presence of “flaws” even when performance is operating within expected parameters
Human operators are almost always in “close physical and temporal proximity” to potential failures, as failure is always just around the corner
Post-accident attributions of “root causes” don’t recognise a nuanced complex sociotechnical understanding of how such systems operate
Hindsight bias, such as following incidents, “remains the primary obstacle to accident investigation, especially when expert human performance is involved.”
All ambiguity in risk and performance “is resolved by actions of practitioners at the sharp end”, whether fortuitous or not.
Human practitioners “are the adaptable element of complex systems”. It’s humans that dynamically adapt on a moment-to-moment basis – not formalised management systems
Thus, people create safety (but also contribute to its demise)
However in saying that, safety is a characteristic of system interactions and not of their components. Safety “cannot be purchased or manufactured; it is not a feature that is separate from the other [functions of a system].
On the above, this means that “that safety cannot be manipulated like a feedstock or raw material”