SafetyInsights.org

This explored the effectiveness of incident reporting systems (IRSs) for improving patient safety and on effectiveness on learning. The relationship between incident reports & actions were evaluated on changes in practice and whether the changes involved settings, processes or outcomes.

Further, single-loop learning (correction of operational issues without significantly changing the overall structure or beliefs) and double-loop (questioning and updating beliefs, structures and practices based on experience) were used to evaluate learning potential of the incident reports and responses.

A systematic literature review was undertaken, which included 43 studies.

Some background on incident reporting was first covered. They note that the detailed information in clinicians’ stories and incident experiences are “reassigned via IRSs into abstract, quantitative variables of the managerial system, thereby reducing the effectiveness of IRSs for learning” (p828). That is, incident reporting systems collapse thick descriptions of the world into neat and simplified, but incomplete, reports. Unsurprisingly, it’s argued that incident systems “do not provide information about the true frequency of organizational errors and are too expensive and bureaucratic” (p828) and may more better reflect the willingness of people to report things rather than the true frequency or true level of system safety.

Interestingly, it’s argued that in healthcare, doctors and surgeons may be reluctant to use IRSs because they see “IRSs as a managerial encroachment on their professional status and individual autonomy (p832) and may further avoid reporting because of fear of litigation.

Of further interest is the argument that incident reporting systems rather than facilitate learning may act to decontextualise knowledge and “act as a structure for organizational power by engendering conflict and competition for control over what counts as an error and hence what type of knowledge is legitimate” (p832). On the topic of IRSs as a source of managerial power, they note that while IRSs may be couched in a rhetoric of learning, they may also “be the product of normative and coercive isomorphic pressure … a method of maintaining and/or restoring an organization’s legitimacy” (p832).

Anyway, enough of the background and onto the findings.

Results:

Overall, this systematic literature review found no strong evidence that IRSs perform better than other forms of reporting. E.g. incident reporting, investigation and remediation didn’t strongly outperform medical chart reviews or other techniques.

Nevertheless, IRSs outperformed other methods when it came to identifying a larger and richer set of contextual information and, surprisingly, required fewer resources than some other methods.

A whole bunch of healthcare/medical specific findings were evaluated but I’ve skipped over these. Of importance however is that while incident reporting can improve clinical settings & processes, found was “little evidence that they ultimately improve outcomes or enable cultural changes” (p826). That is, despite all of the resources invested into incident reporting this study couldn’t find strong compelling evidence that things significantly improve due to them.

For learning potential, evidence was found for single-loop learning (in the form of immediate changes to clinical processes, procedures or settings as a consequence of learning from the IRS), little deep and sustainable change in managerial factors involved in incidents was observed, nor a recalibration of beliefs around incident & performance factors (double-loop learning). However, the authors inform that this absence of evidence for double-loop learning may not be a real absence but could be related to the original studies lacking the methodology to measure double-loop learning.

Of interest is a discussion around the level of focus of incident reports. 18 of the studies in the sample repeatedly refer to making reporting less punitive or comment on ensuring the absence of a blame culture or fear of reprisals. Interestingly the authors argue that these frequent calls for a just culture approach may actually contribute to a sharp-end front-line level focus by continually framing incidents around people rather than up and out at a systems level [* If I understand their argument correctly…]

As a final point, it’s argued that, obviously, learning needs to be genuine rather than rhetorical or espoused but in saying this, IRSs may be driven by an audit culture whose agenda “may be (perceived to be) the reassertion of management control and with the possibility that an IRS exists (or is perceived to) for the purpose of surveillance” and thereby invalidating the espoused belief of learning (p853).

Authors: Charitini Stavropoulou, Carole Doherty and Paul Tosey, 2015, Milbank Q.

Study link: https://dx.doi.org/10.1111%2F1468-0009.12166

Link to the LinkedIn article: https://www.linkedin.com/pulse/how-effective-incident-reporting-systems-improving-ben-hutchinson/?published=t