We chase certifications, implement systems, but what impact are we really having on operational performance? Is achieving certification, like ISO 18001 or 45001, really about improving the governance, assurance, or performance of systems?
Or is it more about window dressing and legitimacy, being seen to be doing the right thing?
Today’s pod covers: Dyreborg, J., Thorsen, S. V., Madsen, C. U., & Hasle, P. (2024). Effectiveness of OHSAS 18001 in reducing accidents at work. A follow-up study of 13,102 workplaces. Safety Science, 177, 106573.
Make sure to subscribe to Safe As on Spotify/Apple, and if you find it useful then please help share the news, and leave a rating and review on your podcast app.
I also have a Safe As LinkedIn group if you want to stay up to date on releases: https://www.linkedin.com/groups/14717868/?lipi=urn%3Ali%3Apage%3Ad_flagship3_detail_base%3Bhdg8uJYYT%2BmsMqZvpHBmdQ%3D%3D
Shout me a coffee (one-off or monthly recurring)
Spotify: https://open.spotify.com/episode/78yaVpW7a6PnhI9nMBq6oC?si=yvAJ_UKCSW2xdKVgZkRHtw
Transcript:
We chase certifications, implement systems, but what impact are we really having on operational performance? Is achieving certification, like ISO 18001 or 45001, really about improving the governance, assurance, or performance of systems, or is it more about window dressing and legitimacy, being seen to be doing the right thing? Let’s find out.
Hello everyone, I’m Ben Hutchinson and this is Safe As, a podcast dedicated to the thrifty analysis of safety, risk, and performance research. Visit safetyinsights.org for more research. Today’s study is from DyreBorg et al. 2024 titled “Effectiveness of OHSAS-18001 in Reducing Accidents at Work – A Follow-Up Study of 13,102 Workplaces.” It’s from Safety Science. The study evaluated the effectiveness of adopting certified health and safety systems, particularly 18001, among 805 adopters versus 12,000 matched controls or non-adopters. Using a long-atutant design over 2010 to 2018, they used government reportable incidents. Providing some background, some research has highlighted a positive difference for organizations who adopt certification. Noting adopters are more likely to enforce health and safety rules and procedures than non-adopters, and the level of safety training was also high among the adopters. Another study found that the adopters fared better than non-adopters in terms of the relative number of safety violations identified by the applicable health and safety regulator. Other work suggested that adopters seem to have a higher overall level of health and safety efforts, but cross-sectional research designs really rule out any causal direction. One Spanish found that positive effects of adopting a certified system where accident rates seemed to decrease, and another Spanish study found a weak relationship between certification and accident rates. One driver of seeking certification might be legitimation. For instance, implementation might be more focused on increasing legitimacy among important stakeholders, and an improved public image as the primary driver for being certified. And in some certified organizations, this underlined driver for legitimacy, being seen to be doing the right thing, might actually counter or limit the effectiveness of these certified systems. In fact, other research has called this legitimacy logical motive window dressing, that is, being seen to care more about safety by becoming certified. Other work might actually call this symbolic safety or demonstrated safety.
So what was found in this study? Well key findings were that the risk of severe accidents at work was 14% lower among the adopters of certified systems compared to non-adopters two years before the data certification, but no further decrease in the accident ratio could be established three years after the follow-up of being certified. So what this means is that companies with an above average pre-certification safety performance are the ones who largely become certified. So there is some sort of pre-certification selection effect. In contrast, they found no decrease in the accident rate for the reportable accidents three years after being certified. They say that these findings support some other research that, for instance, US certified companies to 18,001, tended to be safer than comparable workplaces before their certification. So the companies that are already safer by that very narrow definition around incidents are the ones who tended to get certified rather than the ones that had worse reportable incident performance. They also know that this is in line with other research that adopters of certified systems perform better than non-adopters, but regarding process-related health and safety efforts and content-related efforts, the results couldn’t support these hypotheses. So what that suggests is, even though there might not be a big difference in portable incidents, maybe certified companies tend to perform better with other sorts of activities, safety efforts and interventions and whatnot. They couldn’t support that. They also observed some evidence that there was a tendency that the introduction of a certified system maybe does improve companies’ procedures relating to the reporting of less severe accidents, but this tendency wasn’t statistically significant in this research. They also discussed practices in large companies who had pre-certification audits with a certification agency. Essentially, this provides feedback on what needs to be changed prior to certification. Essentially, it’s seen as a type of gaming where they’re pretty much setting up systems to be better at working through the certification audit rather than actually being more functional, including they highlight that this longitudinal study couldn’t confirm that workplaces adopting a certified system report less severe accidents than work within a three-year follow-up period. And already before the date of certification, the adopting workplaces had a lower rate of severe accidents at work compared to the non-adopters. So this indicates that adopters have a higher safety performance by reportable severe incidents than the average workplace. In any case, adopters didn’t reduce the risk of severe accidents at work during that follow-up period. These results, quote in the paper, call into question the effects of certified systems as an instrument for improving safety in the workplace.
So what do we make of the findings? This one’s another challenging one. I think recognizing that certification isn’t just about coordinating or describing systems for functional goals. It’s also about a bunch of other goals, like legitimation, window dressing, tendering, being seen to be doing the right thing. So recognizing that we might not always be solving the most important issues when managing these certified systems via our paperwork and systems and processes. What I mean is certification can potentially have some unintended byproducts, shifting more efforts towards managing paperwork and documentations as purely part of certification and away from functional goals. And I think drawing on Greg Smith’s words, we need to connect process with purpose. What are we trying to achieve with the elements of the certified system? And are they actually designed, configured, resourced, and understood how these systems and elements can actually achieve those goals? And importantly, we have to be brave to shed the processes that aren’t helping us achieve our goals. And I think there’s a clear cautionary tale here, not to mistake certification with being a safer approach or a better, more functional system. For limitations, of course there were several. The old chestnut that I bang on about every time, but incident data, whatever, it’s statistically rare, randomly distributed, difficult to make sense of, rah rah rah. But the findings are still interesting.
SafetyInsights.org 