As a way to bring some balance to the force with all of the critical research I’ve posted lately, which frankly, doesn’t give me much confidence to trust root cause analysis (RCA) methods (I’ve posted about four other studies highlighting the questionable ability of RCA methods to improve outcomes), this empirically studied some of the positive effects.
This studied the effects of root cause analysis tools on auditors’ ability to find the contributing factors involved in financial report misstatements and evaluate the misstatement’s impact across the audit objectives. 147 auditors were randomly assigned to four RCA treatments: No RCA, unstructured RCA, structured 5 whys RCA, structured fishbone RCA.
Also assessed was the impact of structured & unstructured RCA on auditors’ identified causes, internal control and materiality judgements in considering the causal chain and whether auditor cognitive style impacts the efficacy of RCA methods, based on prior findings where a misalignment between individual cognitive style and task structure undermines performance.
Cognitive style was predicted to impact the use of RCA methods. E.g., something like 5 whys will have a greater impact on auditors with “sequential cognitive styles”, whereas methods like a fishbone more suitable for auditors with a “global cognitive style focused holistic, “big picture” consideration of information” (p18).
Results:
In this experimental study, found was that using structured RCA methods are more likely to identify control-related factors of financial report misstatements. Specifically, 58% of auditors in the three RCA groups identified a specific internal control issue as the misstatement’s causal factor, compared to a lower 28% for the unstructured RCA. For the fishbone group specifically this was as high as 77%.
Use of RCA also led auditors to assess the identified internal control issues to be more severe than the issues identified by auditors in the no RCA group. Auditors’ using structured RCA methods were more likely to link financial misstatements to internal control environment issues compared to unstructured RCA auditors.
Here, the control environment is defined as the set of standards, processes and structures that have a pervasive impact on the overall system of internal control. 52% of auditors using a structured RCA method identified a factor related to the control environment activities whereas just 28% of auditors using unstructured RCA did. That is, “participants in the structured RCA condition were more likely to arrive at factors representing fundamental control problems that would have a pervasive impact on internal control … than participants in the other groups” (p27).
The authors discuss that overall, RCA methods prompted additional critical thought and analysis among the auditors. This led to considering more audit evidence and making more and deeper levels of analysis between the financial misstatements: 58% of structured RCA auditors found a deeper factor compared to 16% of unstructured RCA auditors finding a deep root cause.
Expectedly, auditors using RCA methods spent more time on the audit case than the participants not using RCA. For cognitive fit, auditors with a cognitive fit to the method considered the misstatement to be more material compared to auditors with an RCA method misfit.
Compared to not using RCA methods, the authors conclude that “RCA as an audit procedure increases explanation-based reasoning and has the potential to improve audit effectiveness in integrated audits, although the incremental time needed to conduct structured RCA can impact audit time and efficiency” (p34).
However, importantly, it’s argued that companies should consider and take care when selecting one of the RCA methods to ensure auditor flexibility. That is, so that the tool best suits the user’s cognitive style and is selected or adapted to best fit the audit team or consider using multiple methods that help ensure sufficient analysis.
In my view, as with any tool or approach it’s worth considering the concept of what-you-look-for-is-what-you-find: if you think in terms of causal linear chains or internal control issues (or indeed via systems thinking), then you’ll tend to interpret and frame issues as these.
Authors: Todd DeZoort, Marcus Doxey, Troy Pollard, 2020, Contemporary Accounting Research.
Study link: https://doi.org/10.1111/1911-3846.12649
Link to the LinkedIn article: https://www.linkedin.com/pulse/root-cause-analysis-its-effects-audits-judgments-audit-ben-hutchinson/?published=t
SafetyInsights.org 