Another post spruiking yesterday’s compendium of indicator research – link here.
The attached images highlight some further considerations around indicators – no methodology or logics here, just scattered extracts.
· Image 1 highlights that simple lead / lag denominations may not adequately cover the full performance spectrum of indicators. And there’s more detailed analyses than this
· Image 2 places focus on reactive and active monitoring around the risk control system
o Processes and inputs are defined these as “important actions or activities within the RCS that are necessary to deliver the desired safety outcome”) around the risk control system
· Image 3 focuses on a relationship between indicators – highlighting weak as “poor input and poor output”, and relying on luck where “Good output but poor input performance”
· Image 4 talks about criteria for prioritising indicators – with one being “Will the SPI respond quickly to change (when compared to the speed at which the deterioration of the risk control system could lead to an incident)?”
o It also prompts the question on whether changes in the SPI “stimulate actions that lead to improvements?”
· Image 5 also prompts questions around SPIs in the context of risk control systems – suggesting importance of monitoring effectiveness of RCSs, changes or deterioration of RCSs and more
· Image 6 from Kjellen believes that ‘proactive’ gradually changed to ‘leading’, but without appreciating why this change was important
o Further, in his view, a leading indicator is “an indicator that changes before the actual risk level has changed”; highlighting how indicators should be calibrated to sources of loss and, ideally, provide notice of impending failure
· Finally, several sources list ideal or necessary characteristics of indicators (not shown here), like validity, sensitive to change, comprehensible, practical, actionable, important and more; many indicators in use likely fail many of these characteristics
Most indicators in use seem to measure outcomes. Process safety and quality seem to do better at focusing on precursors, inputs and processes, but this is easier in engineered and/or more linear systems.
Conversely, there may be significant challenges measuring vital aspects of social systems & interactions (‘not everything that matters can be measured’ and yadda yadda yadda).
Overall, and my own bias, I still think Leveson and the systems approaches have a more useful approach to indicators – but note the challenges with social systems.
Check out the compendium via this link.
Refs:
Image 1 – Bayramova et al. 2023. Journal of Safety Research
Image 2 – Health and Safety Executive 2006
Image 3 – Stacey 2012. ICMM
Image 4/5 – RIISB 2016
Image 6 – Kjellen 2009. Safety Science
SafetyInsights.org 