This brief discussion paper explored some limitations of different safety performance indicator types based on an extensive review of the literature.
Based on their review, they identified and discussed three indicator types which are based on a safety management system (SMS) model of measurement: outcome, audit and leading indicators.
Outcome measures:
Authors discuss that the strong drive in the construction industry for safety derived from investigating accidents. This obsession has “led to a strong preference for recording accidents as a primary tool to measure safety performance” (p2).
They see issues relating to these outcome measures as:
- Providing little info about the underlying causal factors. The indicators provide some insights in a reactive way, where we were, and may help companies direction on where they should do via safety objectives, but unable to provide guidance on how to fulfil those objectives (how to get there).
- Outcome indicators have been criticised for being historical in nature, directing attention to solving issues only after an accident has occurred. Authors argue that “a safety indicator is of little use when it is unable to provide early warnings prior to accidents” (p2).
Importantly, managing safety requires a lot of foresight, whereas outcome indicators in their review focus primarily on hindsight. This results in outcome indicators not helping to generate enough insights about prospective failures, where “Foresights are derived from thorough understandings of complex safety phenomenon” (p2).
- Outcome indicators emphasise the absence of safety rather than its presence. Using a quote from the paper, “If we are in the business of promoting OHS, why do we use failures as the measure of our success?” (p3).
On this, they argue that while measuring the presence of safety is difficult, with perhaps more confounding and ambiguous variables which are hard to measure but avoiding the things that are hard will not improve our understanding of safety.
Auditing measures:
Their view of auditing measures is based on the idea that SMSs are evaluated against auditable criteria, which then forms a basis for decision-making. Thus, sensemaking of the SMS is governed in part by the audit criteria’s ability (reliability & validity) to capture the reality of safety.
On this they note that audit criteria are often problematic.
- They are often legal compliance-oriented, which is problematic because being compliant with legislative requirements is “far from sufficient to produce safety” (p3). It may further exacerbate the proliferation of paperwork, where companies are driven to maintain records and traceability rather than functionally managing issues.
- Checking SMSs against standardised criteria, checking the presence of individual practices or components, is based on “the faulty assumption that so long as an SMS is in place and all individual safety practices function effectively, accidents can be prevented” (p3). They counter this argument with the example from Hopkins regarding the Gretley mine accident or from Esso Longford, where SMSs or audits are not sufficient to maintain safety.
Leading measures:
They talk about the shift from lagging to leading. Leading also have many limitations and caveats. They cite seven limitations of leading/positive performance indicators as an example, including things like 1) may not be directly connected to successful outcomes, 2) may be subject to random variation, 3) may encourage over/under reporting, 4) measuring the frequency rather than the quality or impact, plus a bunch more.
Interestingly, drawing on one study they say that “the relationship between [positive performance indicators] and safety outcomes (e.g., injuries) is arbitrary” (p5).
The paper then discussions the overall conclusions. They say that safety performance indicators can be thought of as “proxies for perceiving and understanding the reality of safety” (p6). However, because some indicators are selected on the normative view of SMSs (normative meaning making a statement about how the SMS should be structured and operate), it may be “inappropriate, and even dangerous, to use safety performance indicators that are selected based on the normative SMS approach” (p6).
In explaining this position, it’s argued that while the SMS structure is robust enough, current understanding about how safety is created within complex systems within multiple other frames (HR, cultural frames, political pressures, legal frames etc.) is quite limited in comparison.
They further discuss that current use and measurement of SMS/safety performance is based on fallacious beliefs about why accidents occur and/or how safety is achieved.
For instance, they offer the following statements as reflecting a linear logic which may lead to incomplete risk profiles and an oversimplification of safety phenomena:
- Rules & procedures will keep people safe if only they are followed
- Incidents occur because of worker error and a failure to follow those rules
- Reducing incidents will be driven by improved motivation & training, like motivating people to unquestionably follow the rules
Within a context of incomplete risk profiles & oversimplification of safety phenomena they note that “safety indicators are unable to provide a full picture of safety, which may lead to ineffective decision and action” (p6).
They conclude that safety indicators should mature beyond solely measuring structures of an SMS and particularly from a normative frame. They argue instead that organisations should “first develop a valid description of complex safety phenomenon and then produce scientific knowledge to fill gaps with regard to other frames of safety management systems” (p7).
Link in comments.
Authors: Guo, B., and Tak Wing You, 2013, The 38th Australasian Universities Building Education Association Conference, Auckland, New Zealand.
Study link: https://www.library.auckland.ac.nz/external/finalproceeding/Files/Papers/46530final00039.pdf
Link to the LinkedIn article: https://www.linkedin.com/pulse/how-traditional-construction-safety-performance-fail-ben-hutchinson/?published=t
SafetyInsights.org 