SafetyInsights.org

Drawing on data gathered from a real-life healthcare near incident, three different incident analysis methods are compared based on their abilities to generate system-level outputs and recommendations.

These are:

1) Root cause analysis (5 Whys),

2) HFACS (The Human Factors Analysis Classification System)

3) AcciMap

In describing the basis for the study, it’s said that despite healthcare being “possibly the most complex, socio-technological system … the use of a 20th century linear investigation model [root cause analysis methods] is still recommended for the investigation of adverse outcomes” (p1).

Rather than viewing performance from a linear lens, systems-based approaches view performance as resulting from the interaction within the system; necessitating an analysis of systems as whole entities rather than their parts in isolation.

Thus, this study sought to compare the types of outputs and recommendations generated from the three above methods. 5 Whys was selected as a comparison since it’s specifically recommended by the NHS and was completed in accordance with the NHS guidance material.

Key criteria for evaluation were:  1) the ability to generate graphical representations of the incident, 2) incorporation of data, 3) useability, 4) validity in healthcare, 5) reliability & 6) ability to generate factors at higher levels of a system.

Results

Overall, these findings suggest that both HFACS and AcciMap facilitate recommendations and factors across multiple system levels. See image below for a comparison of methods.

Based on these findings, HFACS “has the most supporting evidence and may be the most implementable” scoring the highest across the six chosen areas, but AcciMap followed closely and would also help generate recommendations that “would offer patients a safer, higher performing, healthcare system” (p9).

In contrast, according to the authors “RCA has been identified as the weakest methodology in this review” (p5). Further, “Its widespread use in healthcare raises concerns that current incident investigation strategies may not provide system level interventions that form the strongest barriers for improving patient” (p5).

The RCA method results placed more focus on front line staff training and policy review – weak solutions to adverse outcomes in a complex socio-technical system.

However, to balance these findings, the authors do opine that “In the real world environment RCA potentially suffers from its own perceived ease of use such that its frequent application by non-experts has weakened its apparent validity” (p5). Despite this and even with investigators trained in the methodologies they still believed it to be the weakest performer.

In contrast, HFACS and AcciMap generated recommendations for both front line staff and higher levels of the system. The authors argue that this focus on higher level controls are more likely to generate stronger barriers to reduce the likelihood of repeat incidents.

Furthermore, they opine that in comparison to the weaker performing RCA method, “the system focus illustrated by HFACS and AcciMap show why a focus on front line staff is unlikely to be of any benefit in preventing repetition of an adverse outcome incident” (p5).

Although HFACS outperformed AcciMap across the six factors, AcciMap was noted to be the only method in this evaluation to enable a whole system view and clearly articulated the regulatory effects on front line incidents. It also readily provides a useful graphical representation of factors and their interactions, which may not be inherently visible. The trade-off is the greater expertise and time needed for AcciMaps.

In concluding, the authors suggest that any of the methods evaluated herein are suitable for healthcare in the hands of expert users, however the inherent linear structure of the RCA method and a lack of particular structure (at least in this context) may result in weaker outcomes.

This is an important point since, according to the authors, “these concerns regarding RCA analysis are supported by evidence of a failure to significantly reduce preventable medical harm over the past 20 years despite its now widespread use in healthcare systems” (p8).

Authors: Peter Isherwood and Patrick Waterson, 2021, Journal of Patient Safety and Risk Management

Study link: http://dx.doi.org/10.1177/2516043521990261

Link to the LinkedIn article: https://www.linkedin.com/pulse/err-system-comparison-methodologies-investigation-ben-hutchinson