This systematically reviewed the evidence on incident reporting systems (IRS) in healthcare. Of 43 studies, 8 compared IRS with other healthcare methods and 35 explored the effectiveness of IRS on settings, structures and outcomes.
It’s a 41-page review, so I’ll only provide some key findings and not the discussion around the findings.
Results
Key findings included:
- No strong evidence was found demonstrating that IRS performed better than other reporting methods
- Although some evidence was found that IRS can improve local clinical settings and processes, they “found little evidence that they ultimately improve outcomes or enable cultural changes” (p826)
- On the above, some evidence was found for single-loop learning in the form of a change in process, but “little evidence of either improvements in outcomes or changes in the latent managerial factors involved in error production” (p827)
33 of the 35 studies could be classified as single-loop learning and not double-loop. Here there was some evidence that IRS could effectively change localised processes and the like.
For double-loop learning, little conclusive or convincing evidence was found for deeper or longer-term changes in governing variables. They note that the absence of evidence here isn’t necessarily evidence of the absence – given the challenges of measuring outputs of double-loop learning, but it does shift our confidence level that IRS are more successful at single over double-loop learning.
Some studies implied that improvements in the “safety culture” had taken place. This included things like changes in risk perceptions and awareness of good practice. Little information was provided on what an improvement in “safety culture” meant, involved or looked like.
Further, none of the studies provided sufficient evidence that any robust changes to double-loop learning had taken place irrespective of claims about “safety culture”.
The importance of an IRS to be genuinely about learning over rhetorical or espoused purposes was stressed. Four studies drew attention to the need for learning to be a core output of an IRS. This was contrasted against a perception of IRSs “being driven by an “audit culture” whose agenda may be (perceived to be) the reassertion of management control and with the possibility that an IRS exists (or is perceived to) for the purpose of surveillance” (p853).
Other methods, like medical charts, had a greater effectiveness in identifying clinical incidents than did IRSs.
For the impact of IRSs on micro and meso changes, few studies reported on robust outcomes and recognised the challenges of actually measuring the impacts since IRSs are often part of a wider program of changes.
In all, it’s argued that:
- Centralised systems, like IRSs, “might not yield the depth of learning anticipated by policymakers” (p854)
- In healthcare, the “imposition of changes generated at the organizational level violates norms of collegiality and self-regulation and creates distrust of managerial motives” (p854)
- Events should be regarded as resulting from complex interactions from across the system instead of just narrowly focused on clinical practice or solvable issues
- To do this, it’s said IRSs need to work across functional, organisational and professional boundaries and “to be contextually located and participative rather than imposed and managed hierarchically” (p855)
- IRSs should “be tailored to local conditions to create a sense of ownership and involvement in efforts toward organizational learning” [** cough, as opposed to trying to standardise every square problem in an organisation into the same round holes]
- Employees need to be confident that an IRS is authentically for learning and not primarily for procedural purposes or an instrument for surveillance
- Finally, they conclude that little evidence exists confirming “that IRSs ultimately improve patient safety outcomes or that single-loop learning changes were sustained, although this may be a consequence of measurement difficulties” and little evidence found that IRSs could lead to double-loop learning in form of deeper changes in structures, cultures or mindsets (p856).
Authors: Stavropoulou, C., Doherty, C., & Tosey, P. (2015). The Milbank Quarterly, 93(4), 826-866.
Study link: https://doi.org/10.1111/1468-0009.12166
Link to the LinkedIn article: https://www.linkedin.com/pulse/how-effective-incidentreporting-systems-improving-ben-hutchinson
SafetyInsights.org 