SafetyInsights.org

This brief discussion paper quickly discussed a few methods used in healthcare for managing variation and quality of care.

They included:

• Incident reporting systems
• Investigations
• Checklists

Providing background, it’s said that:

• Some industries have been particularly successful at managing variability at a systemic and sophisticated level – aviation for instance
• But noting this, variability is the norm in virtually every industry and systems, processes and thus routines emerge to help manage variability
• For some industries, product quality could involve rework or rejecting the product or service with little impact downstream; for healthcare, everything done has consequences for the patient
• So whereas some industries and tasks can have clear performance/tolerance limits, the complex adaptive environment and associated variability within healthcare is said to be particularly challenging for patient safety goals
• Further compounding this goal is that “Every individual healthcare recipient has a unique bodily system, which is dependent upon that individual’s state of health” (p157) and thus, “perception of success or failure of an intervention is a subjective experience unique to that individual” (p157)
• Therefore, even if robust care of delivery processes are in place, perceived variability in health outcomes is “difficult to control and we can see why our ability to ensure consistency, prevent variability and achieve expected outcomes does become a challenge” (p157).

Next, they move to the core methods of discussion. Although this focuses on healthcare, in my view it rings remarkably similar to construction.

Are incident reporting systems to identify variation helpful?

Aviation is said to be an industry healthcare should learn from and an example being the introduction of their Aviation Safety Reporting system.

While that reporting system apparently worked for aviation, its translation to healthcare was problematic since the underpinning principles of aviation were not present in healthcare – i.e. a confidential & independent framework developed to support industry-wide changes.

Comparably in healthcare, it’s said “we have decided that the incident or event reporting system must be interrogated by the immediate supervisor and immediate manager, frequently by those without the technical knowledge about the technicalities of the care delivered” (p157).

This can also result in:

• The manager, now responsible managing the incident, may manage the report in order to close it rather than manage the actual issue in practice
• Further, “leakage of information about some incident having occurred within their span of responsibility cannot be good thing” (p157), which may reflect poorly on their perceived management skills
• Learnings not really being shared across the healthcare system, said to be extremely rare where issues are managed via system-wide changes
• Here they note that “Dissemination of this information across units is extremely rare, and dissemination of this learning across organizations or across the industry would be almost unimaginable” (p157)

They also argue that the “interest in counting and reporting the number of reported incidents and events is also very peculiar indeed” (p157). They discuss whether an increase in the number of incidents and events indicates worsening performance or a better awareness of events and reporting, or both?

Does a RCA really get to ‘the root cause?’

They argue here that use of root cause analysis methods in healthcare is “particularly problematic” (p157).

They believe that the deeper one dives into investigation findings, like via 5 Whys or really any other method, the more likely it is that the root [causes] will always be ‘the decision’ that led to the development or creation of the process” and that it’s rare for these investigations to “go further to systemwide vulnerabilities” (p158).

They add that it’s not uncommon for well-intentioned investigations to “come to the conclusion that ‘the intended process was not followed’ and an effective solution is ‘education and training of the operators” (p158). In the case of many investigations, recommendations focus on training and education and less rarely the factors/conditions which set up education or training as a critical factor.

Moreover, “whether the process of finding a root cause will lead to safer patient care has also been questioned” (p158).

It’s said that healthcare delivery systems and their associated tasks, processes etc. are interdependent and thus it’s problematic to take learnings from one investigation and apply elsewhere and expect similar performance.

 In their view, “An in-depth analysis of one system to find the root cause only informs about vulnerabilities in that specific system” (p158).

Do checklists ensure consistency?

Checklists can fill both a popular and necessary function across many industries – and can help to manage variability in some forms and context.

While checklists can ensure consistency the question “is whether the use of checklists in healthcare also improves safety and health outcomes” (p158).

Compared to incident investigations in healthcare, there’s actually some decent evidence that well-designed checklists in certain contexts can and do improve quality outcomes. However as a counter, they note the challenge in reproducing some of the results of these studies.

For healthcare, there’s also been a tendency for using checklists as a tool to facilitate multidisciplinary team discussions; a rather complex social interaction and phenomenon that “has a wider scope than a tick box checklist” (p158).

This raises questions about whether checklists should be used for conversational prompts, which may decrease the usefulness of a checklist in ensuring consistency of processes and safety.

There weren’t really any more points in this section – but they did reference some studies which evaluated checklists for communication prompts (which perhaps I’ll summarise in future).

In concluding, they state:

• It’s important to use appropriate tools and methods to manage variability but “it is important to do so with full awareness of shortcomings that are inherent in these methods” (p158)
• Incident reporting systems should help enable teams to manage the core issues rather than facilitate and pressure them to manage and close the report
• Further, structures should be in place to help disseminate learnings system-wide [*** nothing new with that suggestion, but it’s something that is extraordinarily difficult]
• Investigations and their methodologies (like a range of RCA tools) may help in some circumstances but “trying to find a root cause is not necessarily value adding”. Rather, learning processes should be to identify system vulnerabilities
• Use of checklists can ensure some level of consistency in process – but it’s important, like anything, that they’re used appropriately.

Authors: Arya, D. K. (2020). International Journal for Quality in Health Care, 32(2), 156-159.

Study link: https://doi.org/10.1093/intqhc/mzz129

Link to the LinkedIn article: https://www.linkedin.com/pulse/we-using-right-tools-manage-variation-errors-ben-hutchinson