Our recent study of 44 major or fatal accident reports in the process, chemical, pipeline, mining, oil & gas industries explored how audits failed prior to the accident (according to the investigation).
I’ll cover some findings over a couple of posts. Note that hindsight bias, counterfactual reasoning and second order/double hermeneutics are a recurring and unavoidable theme in our paper.
We divided the findings into nine sub-categories, grouped into four overarching categories.
The first two categories were:
1) Audits involve a failure to understand.
2) Audits involve a failure to act.
1) Failure to understand:
A failure to understand involved audits not including the necessary or appropriate stakeholders and subject matter experts, as in audits at the Husky Superior refinery explosion not including technical SMEs and operators.
Audits also misinterpreted a hazard or issue despite identifying that issue. Examples included combustible dusts that were detected during audits but were only seen as food safety issues or didn’t evaluate whether they were combustible.
2) Failure to act:
Some issues that were identified were not adequately relayed by the auditing approach or were not acted on or managed; nor did auditing approaches verify that critical issues were addressed as expected.
An example from Macondo was where prior audit recommendations/ improvements had either deteriorated again or were properly addressed to begin with.
Some audits led to inadequate improvement opportunities, when in hindsight another response may have been more suitable. For instance, a phosgene feed system was found not to be using the correct hose, but the audit used a ‘recommendation’ rather than mandatory improvement.
Other examples included issues being raised without improvements, or serious deviations or findings being rejected without formal risk assessment.
Comparing the current findings with those of our previous audit study, it’s clear that audits can over-prioritise the collection of documents and the presence of artefacts at the expense of probing system functionality.
Said differently, a gap exists between how systems are expected to function versus how systems do function. That gap is argued to drive decoupling, further widening expectation versus reality while providing a false veneer of safety.
Further, many audits exhibited a “comprehensive shallowness,” delving excessively into minor system details and paperwork rather than addressing critical factors.
Nevertheless, one plausible interpretation of these findings is that, in the clarity of hindsight, critical aspects amid a sea of issues become clear and evident, embodying “delusional clarity.” (E.g., it’s easy to point out the problems after the smoke has cleared, but harder before the event).
Study links:
Study: https://doi.org/10.1002/prs.12579
Tabulated study findings: https://aiche.onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1002%2Fprs.12579&file=prs12579-sup-0001-TableS1.docx
SafetyInsights.org 
2 thoughts on “Major accident audit failures: a failure to understand and a failure to sufficiently act”