The fallacy of relying on rules for robust risk management in complex high-risk environments

A few extracts from chapter 11 in Foundations of Safety Science by Bergström and Dekker I found interesting.

Here they discuss research in healthcare how:

· Nursing was found to have some 600 rules specifying a ward nurses daily work

· But, nurses could recite just 2-3 of the 600 rules that “supposedly specify their job”

· Despite this, they could effectively and safety perform their duties each day

· Indeed, somehow magically ensuring compliance to all 600 rules would likely make it “entirely impossible to even begin to do the job of nursing”

· Then in anesthesiology, some 4 million operating room standards of practice have been found covering anesthetic work

· Hence, “such a plethora of rules that a zealous practitioner (studying 40 hours a week) would have to spend 2,000 years to read it all (Johnstone, 2017)”

· Therefore, “you cannot derive the full equation set for a complex system. You cannot specify its workings through a coherent, consistent set of rules. You would be writing thousands, millions of rules (which then take 2,000 years to read, as with anesthesia)”

Then they move on to broader insights within complex systems:

· And even with all of these rules, “they would not exhaustively specify how work actually gets done inside that complex system. Many of the rules would contradict each other, or not be relevant or at the right level for the context in which they supposedly apply, or would simply have become obsolete because the field has moved on since the rule was written”

· “Complex systems can never be fully specified—not only because they are complex, but because they are dynamic”

· They highlight how complex systems don’t fare well under a central authority, in part because the central authority “would need to have a stable model of the whole complex system internalized”

· “That would make the authority as complex as the system itself, which would mean that the system could actually not be complex”

· Complex systems are open systems, where they interact with the environment, where the environment “literally ‘folded in’, touching people everywhere in the organization”

· “Complexity does not lead to anarchy and disorder”, but rather repeated patterns and “produce and encourage horizontal, reciprocal self-organization”

· “Feedback loops are an emergent aspect of complex systems”, but this doesn’t mean they’re free to do what they want. Rather, because of “coupling and reciprocity, they are really quite constrained