Bad apples, counterfactuals, and a focus on imagined systems – extracts from DOE accident analysis manual

REALLY cool read from the Department of Energy’s Accident Analysis Manual.

Progressive and integrative of different concepts (eg. barriers, human factors, S-II, bad apples, counterfactuals).

Nothing new for most, but some extracts:

· They cover barrier analysis, highlighting how these questions should be upacked:

o Barriers that were in place and how they performed

o Barriers in place but not used

o Barriers not in place but were required

o Barriers that if present or strengthened, could have prevented the same or similar incident from occurring

· How barriers perform, e.g. effectiveness, availability, assessment and interpretation

· Barrier types – physical, functional, symbolic and incorporeal

· While adding multiple layers of barriers and safeguards “can appear to add more confidence”, it may “also lead to complacency and diminish the ability to use and maintain individual barrier layers”

· They delineate individual vs system accidents, the latter being an “unanticipated interaction of multiple failures in a complex system. This complexity can either be technological or organizational, and often is both”

· WAD vs WAI – noting that there’s always a gap and the expected variability in how people execute their activities

· “a systematic investigative process helps to understand first “what” the variation is and second, determine “why” the variation exists”

· Continuous improvement and learning helps to inform on the work performance gap and where possible, reorganise and minimise

· Investigations may fall into the trap of ‘system optimism’, which is “is the belief that systems are well designed and well maintained, procedures are complete and correct, designers can foresee and anticipate every situation, and that people behave as they are expected to or as they were taught” (e.g. WAI)

· Conversely, ““System Reality” is the belief that things go right because people learn to overcome design flaws and functional glitches, adapt their performance to meet demands, interpret and apply procedures to match conditions, and can detect and correct when things go wrong”

· Challenges with the selection (‘construction’) of causality – “Cause and effect relationships are normally inferred from observation, but are generally not something that can be observed directly”

· “Accident investigations, however, involve the notion of backward causality, i.e., reasoning backward from Effect to Action”

• “The Bad Apple Theory is based on the belief that the system in which people work is basically safe and worker errors and mistakes are seen as the cause of the accident. An investigation based on this belief focuses on the workers’ bad decisions or inappropriate behavior and deviation from written guidance”

• “From the investigation perspective, knowing what the outcome was creates a hindsight bias which makes it difficult to view the event from the perspective of the worker before the accident. It is easy to blame the worker and difficult to look for weaknesses within the organization or system in which they worked. The pressure to find an obvious cause and quickly finish the investigation can be overpowering”

• A haphazard use of counterfactual reasoning within investigations – “Using the maze metaphor, what was complex, with multiple paths and unknown outcomes for the workers, becomes simple and obvious for the investigator. The investigator can easily retrace the workers path through the maze and see where they chose a path that led to the accident rather than one that avoided the accident”

• These include statements like could have, should have, would have, if only they had

• “ The problem with counterfactuals is that they are a statement of what people did not do and does not explain why the workers did what they did do. Counterfactuals take place in an alternate reality that did not happen and basically represent a list of what the investigators wish had happened instead”