Are our investigations blinded to the functioning and effectiveness of risk controls? Are our current approaches, and mental models about how safety events occur, defined less by what they unpack and more by what they leave in the dark?
This study unpacks these questions, and evaluates how accident investigators consider, or not, the functioning of risk controls within the context of investigations.
Ref: Dodshon, P., & Hassall, M. E. (2017). Practitioners’ perspectives on incident investigations. Safety science, 93, 187-198.
Spotify:
Shout me a coffee (one-off or monthly recurring)
Transcription:
Have you ever looked at a recurring workplace incident and thought, “Didn’t we investigate this already?” It’s a frustrating sense of deja vu, isn’t it? We pour effort into understanding what went wrong, yet the same kinds of problems keep resurfacing.
This week, we’re uncovering how, if our tools and mental models are only designed to find specific kinds of problems, like human error, then that’s all we’re ever going to find. This has been called “What you look for is what you find.” Therefore, what if our investigations are defined less by what they find, or socially construct as causal, but more by what they leave in the dark?
Good day everyone. I’m Ben Hutchinson. This is Safe As, a podcast dedicated to the thrifty analysis of safety, risk, and performance research. Visit safetyinsights.org for more research.
Today’s study explored how incident investigations approach and fail to learn from incidents, including the types of controls implemented after investigations. It’s titled “Practitioners’ Perspectives on Incident Investigations” by Dodgson and Hustle (2017) in Safety Science.
What were their methods?
222 survey respondents were included in this data across mining, construction, transport, postal, warehousing, and other industries. They also reviewed incident investigation manuals and procedures.
Oh hey, future Ben here, speaking after this episode has been recorded. Now, despite knowing better, I’ve used the terms “barrier” and “control” synonymously in this episode. Even though, to be precise, barriers and safeguards are both types of controls. So controls can be considered a high order category. I’ll cover more precise definitions of controls, barriers, and safeguards in a future podcast. But using this paper’s definition, a control is an act, object, or system intended to arrest or mitigate an unwanted event and that a control must be specifiable, immeasurable, and auditable.
So what did they find?
Regarding the identification of controls present at the time of an incident, just over 57% of all industry survey respondents indicated that the extent of controls being present at the time of the incident was considered in their investigation process. This means that around 57% of the respondents typically look at the barriers or controls during the investigation.
In mining, it was around 50% of investigations looked at the barriers, whereas in construction, up to 73% of investigations always considered whether the controls were in place. In transport, postal, and warehousing, it was around 64.5%. This suggests that most organizations are at least asking the basic question: were safeguards in place when this incident happened?
For assessment of the effectiveness of those controls, around 57% of the respondents said that control effectiveness wasn’t always considered as part of the investigation process. Similarly, 60% of mining and close to 60% of construction indicated that control effectiveness wasn’t considered in their investigation process. In contrast, 58% of transport indicated that control effectiveness was always considered.
So, while many check if controls were present, a majority have found to ask the crucial follow-up question: did that control actually work as intended?
Next, they looked at absent controls that if they had been present, they might have prevented the incident. Across the whole sample, just over 50% indicated that absent controls weren’t always considered part of the current analysis. It was around 57% in mining and other groups around 59%. So the consideration of controls being absent at the time of incident wasn’t always considered. In construction, around 55% indicated that their investigation always identified absent controls, that if present, may have been effective to help prevent or mitigate the event.
This highlights a significant missed opportunity for learning. Many investigations aren’t asking: what else could we have done that wasn’t there at the time?
Regarding opportunity to learn, only around 61% of investigations focus on controls. Less than 42% focus on evaluating both present and absent controls, and just 34% of incidents assess or make recommendations about enhancing the effectiveness of controls. This means that while incidents happen, a surprisingly low percentage of investigations are actually getting to the core of control effectiveness, which is vital for real learning.
Next, they evaluated the incident manuals. One finding was that the most frequently used tools by investigators don’t routinely require arresting and mitigation controls to be identified as part of the incident investigation process. In practical terms, the very forms and checklist investigations used often don’t prompt them to think about how to stop or lessen the impact of an incident.
Another interesting finding was that only 14 of the 24 investigation documents that the researchers evaluated actually required a description of the incident or required any human factors considerations or the sequence of events be part of the investigation. I think this absence of human factors integration, at least in the documentation, to be quite telling. This suggested foundational elements for understanding the how and whys of incident and performance, routine and un-routine performance, and particularly human element are often missing from the procedural blueprints of investigations.
They found very little reference to the role of mitigation controls, although prevention controls were more readily recognized. So investigators were comfortable looking for prevention controls, but very rarely the mitigation controls. Mitigation is quite a big area of research within the resilience engineering world. It’s also part of failing safely, that when disaster occurs, how well are we set up to succeed?
They also noted that the initiating events or the initiating hazards was neglected by over 60 percent of the organization’s incident investigation documentation. This means organizations are frequently documenting what happened and the contributing factors, although very narrowly defined, but are often overlooking how it started or how to contain it after it begins.
So what are some limitations?
For one, it’s a survey, so the depth of findings are somewhat limited. It’s also unclear how representative the findings are. They note that while the mining sample was really large, the other sectors weren’t so.
So what do we make of the findings?
As the study found, some existing investigation approaches and tools weren’t that good at directing people to understand the context of incidents, or, likely by extension, understanding hazardous work, normal, daily hazardous work.
A focus on barrier performance also wasn’t a big focus in investigations. This is problematic if your worldview and your approaches are based around barrier performance, for instance, in the critical control management frameworks or similar frameworks.
As a way to improve the evaluation of real, daily work and the effectiveness of our barriers, why not try these sorts of questions?
- What barriers were expected to be in place? And what barriers were actually in place?
- Did the barriers perform as expected? If not, how?
- Were the barriers and their performance specifications even known by stakeholders?
- What happened last time this task was done? Were the barriers used?
- Or how is this task normally done? Or what does good look like? What’s good recognized industry practice?
- Or what about some questions from HOP or safety tool approaches?
- What makes it difficult to apply these controls?
- When was the last time you had to work around the barriers?
- What’s critical for ensuring safe and reliable work of this kind?
- If I gave you a thousand dollars to improve the work, how would you spend it?
That’s it on Safe As. I’m Ben Hutchinson. If you found this useful, then please share, rate and review and check out safetyinsights.org for more research.
SafetyInsights.org 