Why safety performance indicators?

A response from Andrew Hale to Andrew Hopkins’ 2009 article about indicators (link in comments).

[See tomorrow’s compendium dedicated to Hopkins & Hale]

Extracts:

· Hale starts with outlining why we need indicators, suggesting three obvious roles

· Monitoring the level of safety in a system, which “answers the question: is the level of safety OK as we are now managing things”?

· Deciding where and how to take action, and this “requires indicators deeper in the system showing the state of those causal links to the harm which have been proven to exist (or at least are strongly believed), so that we know (or believe) that manipulating them will result in the system becoming safer”

· “Motivating those in a position to take the necessary action to take it. This requires indicators which those persons see as being relevant and as being influenceable by themselves”

· For Rasmussen, “Relying on reactive (lagging) indicators to guide action is the ‘fix and fly’ approach, as opposed to that now universally regarded as essential for the major hazard industries, which need to predict and act before the disastrous event occurs”

· Hale discusses a 2008 study he conducted, where the highest performing organisations with the largest accident drops were the ones that “had KPIs for their managers which were not only directed towards lowering the accident or absence rate, but also at driving up the intermediate indicators such as reports of dangerous situations and their resolution”

· “A related topic is the need to have clear, explicit and well-articulated models forming the basis for defining and using indicators. The HSE guidance document (HSE, 2006) fails badly in this respect”

· For one, it presents the Swiss Cheese concept “with the idea that an indicator is leading or lagging in respect of the working of a barrier or Swiss cheese slice, rather than the much more commonly used definition that it leads or lags the occurrence of harm”

· He argues that the distinction isn’t always between black and white leading and lagging, but “between degrees of leading. In other words there is a continuum …represented, for example in the bowtie model … from extreme leading to extreme lagging”

· Moreover, while injury measures are often criticised for mature organisations or the developed world, “Counting the numbers of dead and permanently maimed in a poorly performing system, … in developing countries is a good indicator of safety in that it tells clearly that action is needed”

· Hale suggests 6 criterion for robust indicators: valid, reliable, sensitive to change, representative, openness to bias, cost effective

· “One of the greatest problems with indicators which are used in practice as the basis of rewards or punishments, as Hopkins discusses in the last section of his paper, is that managers learn to manipulate them and so contribute to the false sense of security”