The content on this page has been converted from PDF to HTML format using an artificial intelligence (AI) tool as part of our ongoing efforts to improve accessibility and usability of our publications. Note:
- No human verification has been conducted of the converted content.
- While we strive for accuracy errors or omissions may exist.
- This content is provided for informational purposes only and should not be relied upon as a definitive or authoritative source.
- For the official and verified version of the publication, refer to the original PDF document.
If you identify any inaccuracies or have concerns about the content, please contact us at [email protected].
Thematic Review: ISQM (UK) 1 Root Cause Analysis (RCA)
Thematic Review: ISQM (UK) 1 Root Cause Analysis (RCA)
September 2024
The FRC does not accept any liability to any party for any loss, damage or costs howsoever arising, whether directly or indirectly, whether in contract, tort or otherwise from any action or decision taken (or not taken) as a result of any person relying on or otherwise using this document or arising from any omission from it.
© The Financial Reporting Council Limited 2024 The Financial Reporting Council Limited is a company limited by guarantee. Registered in England number 2486368. Registered Office: 8th Floor, 125 London Wall, London EC2Y 5AS
1. Executive summary
A root cause is the core factor – the highest-level cause that sets in motion a reaction that leads to a positive or negative outcome. Root Cause Analysis (RCA) is the process of uncovering these core factors to identify action(s) that would prevent negative outcomes recurring and promote positive ones. Tier 1 firms have been using RCA for many years to determine the drivers of audit quality findings and it has now become a requirement under the International Standard on Quality Management (UK) 1 (ISQM1 (UK) 1.
RCA can play a key role in evaluating findings and driving ongoing, proactive quality management.
ISQM (UK) 1 is a principles-based standard. It requires firms to develop a customised and iterative system of quality management (SoQM) to identify and respond to their quality objectives and risks to those objectives, and to identify and remediate findings and deficiencies on a timely basis.
The nature, timing and extent of any RCA procedures are expected to be driven by a firm's facts and circumstances, including its size and complexity, and by the nature and significance of the matters being analysed. Therefore, the development of an RCA programme should be proportionate to the firm and the audit engagement and firmwide findings the firm is analysing.
ISQM (UK) 1 requires firms to:
- Design monitoring processes that can provide timely information about the design, implementation and operation of its SoQM and identify deficiencies.
- Evaluate findings from this monitoring to identify deficiencies.
- Perform RCA on all identified deficiencies to assess their severity and pervasiveness and design effective remediating actions.
- Design, implement and operate their SoQMs to proactively manage the quality of their engagements.
Smaller firms, or those undertaking more straightforward audits, may be applying RCA to less complex deficiencies and have simpler SoQMs. Therefore, their RCA procedures are likely to be less complicated and less formal.
However, the standard does not set out how to evaluate findings, proactively manage quality or perform RCA over deficiencies.
During the 2023/24 inspection cycle, the FRC reviewed Tier 1 firms’ ISQM (UK) 1 implementation. As RCA is both required for deficiencies and a key tool for proactively managing audit quality, understanding the firms’ RCA processes is integral to our approach to monitoring the firms, as well as our forward-looking supervision. We supervise how firms manage the effective design, implementation and operation of their SoQMs, and proactively manage their audit quality. We also assess whether their quality plans and initiatives are based on a robust understanding of why quality findings arose and how they may be addressed.
The FRC undertook this thematic analysis of RCA across the six audit firms in Tier 1 to support our supervision of ISQM (UK) 1, particularly the principles in the Monitoring and Remediation component.
We reviewed the firms' policies, processes and procedures for the following overarching areas relevant to undertaking an RCA programme:
- Scoping RCA programmes.
- Selecting, training and deploying personnel to perform RCA.
- Undertaking RCA and identifying themes.
- Monitoring the effectiveness of these RCA programmes.
The FRC is an improvement regulator and acknowledging that ISQM/ (UK) 1 does not set out how to perform RCA, we have identified eight elements of an effective RCA programme. In this report, for each overarching element, we have explored how the Tier 1 firms have approached the principles of the standard and shared observations and good practice. This is intended to support firms in considering how to design, implement and operate their own SoQMs.
We were pleased to see the extent to which firms have continued to invest in, and develop, their RCA programmes for audit engagement findings.
We expect audit firms to consider these concepts and examples in the context of their nature and circumstances. We do not expect all observations and good practice to be appropriate for all firms.
We were surprised by the variation in the scope and formality of firms' RCA programmes for findings and deficiencies identified in their SoQMs, and for other ethics and firmwide matters that could indicate SoQM concerns. Several firms could expand the scope, methodology and resources for this RCA to enhance its robustness and consistency, and better understand where SoQM improvements are needed.
Smaller firms may find it most useful to focus on the eight elements described in the left-hand column of pages 6-8. The remainder of the report provides further details on each element for firms that wish to mature their RCA programmes.
We did not perform sample testing over RCA results or the planning or implementation of remediating actions.
Throughout this publication, the following symbols are used:
- Represents good practice identified.
- Represents observations noted.
2. Summary of key messages
Factors to consider when designing and implementing an RCA programme
Scope – firmwide
- Firms are required to perform RCA over all deficiencies in their SoQMs.
- Firms should assess where RCA over other findings (for example, exceptions in internal policies, processes and controls, and survey results) would support identifying SoQM deficiencies. This may depend on the complexity and nature of the other findings.
What have we seen? * The extent and formality of firms' RCA for ISQM (UK) 1 deficiencies varied, with some performing limited RCA for some deficiencies. One firm had not yet implemented its RCA process for deficiencies.
What good practice did we see? * One firm performed RCA for all ISQM (UK) 1 findings, as well as a wide range of other internal firmwide matters.
Scope – audit engagement findings
- Firms should consider where RCA would help assess if audit engagement findings and Prior Year Adjustments indicate SoQM deficiencies. Firms should consider what coverage would enable this based on the number and range of matters.
- Firms may consider where RCA over good practice points would aid proactive quality management. This may be relevant if good practice and findings arise in similar areas.
- Firms should be alert to changes in their circumstances that should impact the planned scope of their RCA.
What have we seen? * All firms performed RCA over all external audit engagement findings and good practice points. Most firms performed RCA over all internal audit engagement findings and some good practice points. * Four firms perform RCA on PYAs, to differing extents. * Five performed thematic reviews, though the number and topics varied.
What good practice did we see? * One firm performed RCA for all PYAs on public interest entities (PIEs) or impacting profit. Another firm performed thematic RCA over PYAs on ongoing audits. * Two firms had structured mechanisms to track emerging issues and identify where thematic reviews should be performed, and which findings should be scoped in.
Resources
- For all RCA undertaken, firms should ensure relevant individuals have sufficient capacity, skills and experience, in line with the complexity of the RCA procedures.
- Where RCA is undertaken, firms should consider what reviews and oversight will support robust and consistent processes.
What have we seen? * Firms' central RCA teams perform RCA on audit engagement findings and, often, SoQM matters. Three firms' SoQM teams also perform RCA. * Some firms use their Ethics or Legal teams for certain RCA projects. * Two firms use staff from the Audit practice to resource RCA work.
What good practice did we see? * One firm uses Audit partner candidates to perform RCA, allocating them based on sector experience. * At one firm, the results of all RCA are reviewed by an independent challenge panel.
Training
- Training on technical matters, soft skills and behavioural and cultural factors can support RCA performance. The extent and nature of training should reflect the complexity of the matters subject to CA.
What have we seen? * All teams undertaking RCA receive technical and soft skills training. * Firms' approaches to training on soft skills and behavioural factors varies. They use a combination of external behavioural specialists and internal culture or business psychology experts. * Some firms also use internal forensics team to provide training.
What good practice did we see? * Two firms hold regular UK-based learning events, as well as sessions with other RCA teams in each firm's global network.
RCA timeline
- RCA for ISQM (UK) 1 deficiencies should be completed in time to support the annual evaluation.
- When planning RCA, firms should consider the need to identify deficiencies on a timely basis. For audit engagement findings, this may mean planning RCA as soon as the findings are finalised.
What have we seen? * All firms have fixed timelines for the completion of audit engagement RCA. * A few firms have set timelines for RCA on SoQM matters. Some firms could improve the timeliness of this RCA to support the annual evaluation process.
What good practice did we see? * Several firms use a wide range of approaches to structure interviews and challenge/corroborate insights that arise.
Data gathering and interviews
- Firms should assess what data would support each set of RCA procedures. This may include:
- Audit hours, levels of overtime, use of certain tools, or the timing of audit milestones.
- Review of audit working papers.
- Interviewing personnel from a range of grades and roles.
- Surveys and questionnaires to gather a range of perspectives.
What have we seen? * All firms have guidance for audit engagement RCA including for data gathering and interviews. Most also have some firmwide RCA guidance. * Firms could enhance their firmwide RCA guidance, including on how the methodology should be tailored for different matters. * The range and number of Audit Quality Indicators (AQIs) used by firms varied.
What good practice did we see? * Two firms developed RCA dashboards to track AQIs and causal factors identified during audits.
Identifying themes
- Firms should consider how to categorise and track causes to identify themes. Establishing a taxonomy is one way to enable this. Where a taxonomy is used, it can be useful to map the categories of causes to the quality objective and risks in the SoQM.
- When using taxonomies, firms should balance the benefits of granularity and aggregation and regularly review and refresh them.
What have we seen? * All firms have taxonomies to categorise and aggregate the causal factors for audit engagement findings. Some also use taxonomies for firmwide causal factors. * Five firms' taxonomies have multiple levels to better enable aggregation, while capturing granularity. * Three firms rank causes to weight them in the aggregation process. Others apply weighting more judgmentally.
What good practice did we see? * One firm set numerous RCA KPIs. Each KPI is scored 1-5 and for KPIs falling below 3, actions are planned to improve the process.
Monitoring the effectiveness of RCA
- Firms should assess the effectiveness of their monitoring processes to ensure they are meeting the requirement of ISQM (UK) 1. Where firms use RCA, they should consider whether their programmes are effectively identifying the drivers of quality outcomes and deficiencies in the SoQM.
What have we seen? * All firms undertake some monitoring of their programmes by: * Tracking the effectiveness of remediating actions and the recurrence of quality findings and causal factors. * Setting and monitoring key performance indicators (KPIs). * Gathering feedback from relevant stakeholders. * Periodic holistic evaluation of RCA programmes. * Firms could increase their monitoring to assess whether their RCA programmes meet their objectives.
3. Scope of RCA
Firms should scope their RCA programmes to support the identification of deficiencies and of actions to proactively manage audit quality. This can be enabled by identifying systemic themes and trends. The scope of firms' RCA varies based on their focus areas, risk factors and resource capacity, and may change over time. Firms should regularly review their scoping to determine whether any changes are needed based on emerging issues. The table below compares firms' RCA coverage of findings and good practice from external and internal audit engagement reviews and PYAs, and their use of thematic reviews, for the 2022/23 cycle only. Firms are not expected to cover all areas as they should base their coverage on an assessment of their circumstances.
Key
- Full coverage
- Scoped coverage
- No coverage
Several firms specifically scoped-in good practice in areas where adverse findings also arose, to understand inconsistent quality. For example, one selected 28% of its 'good practice' audit files, focusing on areas where key or pervasive findings had also arisen, with coverage across service lines and offices.
3.1 Internal and external findings
Firms perform RCA on audit inspection findings to identify how to prevent recurrence. Scoping-in findings from both internal and external reviews can help identify systemic issues. External reviews usually cover audits with a higher risk or profile, while internal reviews cover a larger number and broader population of audits, and different areas of focus.
RCA over positive outcomes can help identify quality drivers that might improve outcomes elsewhere and assess the impact of firms' quality initiatives. Comparing causes of positive and adverse outcomes identified on similar audits and/or audit areas, can also help in the design of effective remediating actions and broader quality initiatives.
For audit files rated as satisfactory or as limited improvements required, less significant findings may still have arisen. Firms could consider performing RCA over these findings to understand causal factors that may lead to key findings on other audits.
1This firm did not perform RCA over all adverse findings, due to capacity and timing constraints. Instead, it performed thematic reviews on a range of recurring internal findings. As a result, this firm performed the highest number of thematic reviews. The firm has increased its resources and aims to extend the scope of its RCA over internal findings.
3.2 Prior Year Adjustments (PYAs)
PYAs are recorded in financial statements where a correction is needed to historic information due to errors by management or the auditors.
Four firms performed RCA on PYAs, to differing extents. Three set fixed criteria for which PYAs they covered, for example listed companies. Another firm only performed RCA for one PYA in the cycle, due to capacity constraints. However, it is planning a thematic RCA review over recurring themes arising on PYAs.
One firm performed a thematic review over PYAs to the cash flow statement. This included PYAs identified on ongoing audits, which allowed systemic causes to be identified, and remedial actions designed, quicker.
One firm performed RCA for all PYAs on PIEs and profit-adjusting PYAs on other entities. In addition, it reviewed the full PYA population for emerging themes, where further RCA may be beneficial. This firm also specifically scoped-in a sample of PYAs for non-PIEs.
Five firms performed thematic reviews, although the number and topics varied significantly. Firm A performed the highest number as it also use these reviews to cover internal audit inspection findings. The remaining firm assessed whether thematic reviews were needed, but did not identify any necessary topics in the 2022/2023 cycle. Three firms have a set process, with defined criteria, to pick thematic topics. The other firms' approach is purely judgmental.
Examples of recent thematic reviews performed relate to: * Director-led audit engagements with adverse findings. * Use of experts, specialists and consultants. * Audit of impairment, going concern and revenue/receivables. * Accounting judgements and disclosures. * First year audits.
At one firm, topics are based on areas with adverse findings on 10% or more of audit reviews, considering whether the findings are recurring and significant, and if they arose on significant risk areas. The firm selects 60% of the relevant audits, per topic, including all non-compliant engagements. The fixed threshold supports sufficient coverage.
Another firm iteratively tracks a range of inputs, including significant audit findings, emerging concerns from reviews of ongoing audits and consultations, and corporate reporting findings. This supports the timely identification of key topics. The firm also looks for themes in sub-populations, such as certain offices, business units or engagement leaders with a particular background or level of experience. It then selects 10% of the audit files with findings in the relevant areas.
3.3 Thematic RCA reviews over Audit file findings
In thematic reviews, firms perform RCA on certain types of findings found to be common across multiple audits, to understand why they occurred and develop remediating actions relevant for multiple audits. Topics are often driven by key and emerging areas of concern, as well as findings that recur year-on-year, to determine if the causes are consistent and why previous remediating actions were not effective.
3.4 Firmwide RCA
Firms also carry out RCA on findings from firmwide matters, including findings and deficiencies identified under ISQM (UK) 1. The scope of their firmwide RCA, in the 2022/2023 cycle, is summarised below. Firms are not expected to cover all areas as they should base their coverage on an assessment of their circumstances.
3.4.1 External and internal firmwide findings
External findings relate to regulatory reviews of firms' processes, such as acceptance and continuance. Internal firmwide findings may include conduct concerns, risk management and compliance exceptions, learning and HR matters, whistleblowing reports, complaints, AQIs or survey results. RCA can help identify where these indicate systemic issues and what remediating actions may be effective.
One firm only carried out limited RCA for certain ISQM (UK) 1 deficiencies. Its current policy is only to perform full RCA for severe deficiencies, which may reduce its capacity for continuous improvement. Another firm did not carry out formal RCA for remediated deficiencies as they were simple in nature, relating to weaknesses in automated processes. Going forward, this firm should ensure its RCA, for all deficiencies, is proportionate, based on the nature of the deficiencies.
One firm plans to increase its RCA scope to include findings.
One firm performs RCA for all ISQM (UK) 1 findings to support its assessment and remediation of all findings.
One firm has extensive RCA coverage over internal firmwide findings. This includes from monitoring onshore and offshore resources, survey results, internal compliance breaches, performance management, regulatory reporting, and speak-up and whistleblowing reports (of non-financial conduct matters).
3.4.2 Ethics breaches
RCA can help assess the significance and likelihood of pervasiveness, so firms can focus on breaches that could be driven by systemic or cultural issues, and assess the risk of further breaches. RCA can also help plan actions to reduce the risks of recurrence.
Firms consider trends and recurrence when scoping breaches. This led two to perform thematic reviews on non-compliance with personal financial independence rules. Two firms also specifically considered near misses when identifying potential topics for thematic reviews.
Of the firms that do not perform full RCA over ethics breaches, one is considering expanding its scope.
2This firm had yet to implement a process for RCA for ISQM (UK) 1 deficiencies. The firm is addressing this point.
4. How RCA is performed
In this section, we consider which teams firms use to undertake RCA and how they are resourced, trained and allocated to projects.
4.1 Resources
4.1.1 RCA teams
RCA is more robust when it is performed using a consistent approach, by individuals with sufficient understanding of the process and purpose. Where it is performed by those outside of the core RCA function, firms should provide any necessary training, to develop their skills and knowledge. They should also consider whether these individuals are sufficiently independent of the underlying areas or findings. The below table summarises which teams perform RCA in each firm:
Two firms have recently recruited RCA individuals to increase their resource and expand the scope of their RCA programmes. Two firms supplement their core RCA team with a pool of trained individuals from the Audit practice, on a project basis. Individuals are allocated to projects based on their skills and experience.
RCA teams benefit from a mix of seniority and skills. Those performing RCA should feel confident questioning and challenging individuals at all grades, have the audit experience to understand the issues under review, and have strong interpersonal skills. RCA reviewers may be more objective where they are independent from the matters they are reviewing. The backgrounds of core RCA team members across firms include:
| Firm A | Firm B | Firm C | Firm D | Firm E | Firm F |
|---|---|---|---|---|---|
| Audit, risk and quality, forensic accounting, ISQM1 | Audit | Audit, ISQM1, consulting, controls testing, risk management | Audit, hot review, ISQM1, controls testing | Quality support, incident investigations | Audit, technical accounting, tax, digital and culture |
For ISQM (UK) 1 deficiencies, initial RCA by the SoQM team with further review by the Core RCA team. *For more details on extent of ethics breaches in scope for firms refer section 3.4.2.
4.1.1 RCA teams (continued)
At the end of February 2024, the size of firms' core RCA teams ranged from five to nine people, although in some firms these teams also had responsibility for other areas. Teams were primarily composed of managers and senior managers and led by a director or partner. Three firms include an associate or analyst in the team to support with data analysis and co-ordination.
Where RCA teams are led by a director, firms should consider whether those individuals have sufficient experience and authority, as well as appropriate reporting and communication lines with the sponsoring partner/s, so they can seek guidance and support where it is needed.
At one firm, the core RCA team has a data analyst, who has experience in various analytics tools and programming languages. Their work has included reperformance analytics, anomaly detection and development of standardised data visualisation tools for the team.
4.1.2 Drawing colleagues from outside the core RCA team
Drawing personnel from the Audit practice can help provide individuals with the necessary technical knowledge. However, it is important that they are adequately trained on RCA and have time to develop sufficient experience and skills. Some of the benefits and risks include:
| Benefits | Risks |
|---|---|
| * Bringing valuable insights from their work experience. | * Variation in experience, could impact the consistency of RCA quality. |
| * Due to more flexible resources, the ability to increase RCA scoping as needed. | * Inconsistency in the work performed. |
| * In-depth understanding of plausible causes and context. | * Individuals bring biases from their own audit experience. |
| * Offering individuals wider exposure to RCA, so they can bring that knowledge to their own audit work. | * Competing pressures on their time due to ongoing audit work. |
| * Not enough experience to consider behavioural matters/aspects. | |
| * Conflict of interest if individuals undertake RCA for matters where they work on the relevant engagement, or in the relevant team. |
Firms may assess their resource needs and capacity to determine if they need additional personnel so that appropriate individuals can be selected and trained, with consideration of their experience, seniority and independence from the areas and engagements being analysed.
4.1.2 Drawing colleagues from outside the core RCA team (continued)
| Firm A | Firm B | Firm C | Firm D | Firm E | Firm F | |
|---|---|---|---|---|---|---|
| Audit practice | √ | √ | ||||
| Offshore delivery centres | √ | |||||
| Ethics team, General Counsel and other teams | √ | √ | √ | √ |
At one firm, RCA assistants from a fixed pool of audit personnel are selected for RCA review based on a number of factors: * Individuals' availability, experience, seniority and skills. * Severity of the audit engagement grading and the findings. * The nature of the findings and whether specialist or industry knowledge is needed. * Seniority of those to be interviewed. * Whether individuals are sufficiently independent from the engagement and team. * Workload and capacity.
All RCA assistants are senior managers or above and have an audit background. All reviews performed by RCA assistants are reviewed by the core team and the RCA leader.
At one firm, partner candidates from the Audit practice undertake audit engagement RCA. They are allocated to RCA based on their experience and skills. This allows the RCA team to benefit from their seniority and sector experience and enables learning points from RCA to be taken back to the business on a real-time basis.
4.1.3 Allocation to RCA procedures
When allocating RCA team members to reviews, it is useful to assess their experience and seniority, the nature and severity of the relevant findings, and potential conflicts of interest. This is particularly important where personnel from the Audit practice perform audit engagement RCA, or where the SoQM team performs RCA over ISQM (UK) 1 matter.
One firm requires that, where files are rated as requiring significant improvements, the RCA director must lead all interviews, with the RCA partner present in any interviews with the engagement leader. At another firm, RCA firmwide interviews are only performed by core RCA team members (rather than interviewers sourced from the Audit practice), as the team has received ISQM (UK) 1 training.
4.2 Training
We have focused on the extent and type of training provided to individuals and teams performing RCA, as well as the use of specialists to train and develop the RCA team.
4.2.1 Extent and content of training provided
Training usually covers technical knowledge, RCA techniques and principles, behavioural and cultural factors, and soft skills such as asking challenging questions, knowing when to keep probing, and creating a safe space to promote an open and frank dialogue. For those undertaking RCA over SoQM matters, it also usually covers the principles of ISQM (UK) 1 to provide context for the matters being analysed. The extent of training offered to the teams performing RCA at the firms varies.
| Type of training | Firm A | Firm B | Firm C | Firm D | Firm E | Firm F |
|---|---|---|---|---|---|---|
| ISQM (UK) 1 principles and concepts | Core team, Audit practice assistants | SoQM team | Core team | Core and SoQM team | ||
| RCA principles, tools and techniques, data gathering mechanisms | Core team | Core team, Audit practice assistants, Independence team | Core and SoQM team | Core team, Audit practice/offshore colleagues | Core team | Core and SoQM team |
| Behavioural, cultural and psychological matters | Core team | Core team, Audit practice assistants | Core and SoQM team | Core team, Audit practice/offshore colleagues | Core team | Core and SoQM team |
Training is a combination of virtual/e-learnings and in person and is usually on an annual cycle. Additionally, individuals receive the technical audit training given to the Audit practice.
For new joiners to the core RCA team, two firms use formal shadowing programmes so they learn how to undertake the full RCA process and see skills being modelled.
At two firms, the RCA team has UK-based strategy events, workshops and monthly meetings, as well as regular sessions with other RCA teams in each firm's global network to enable learnings to be shared across countries.
4.2.1 Extent and content of training provided (continued)
All firms have provided training to RCA teams on behavioural factors and soft skills, which are important for creating a safe space for discussion and effective challenge. Firms have taken a variety of approaches to developing these skills, with examples highlighted below:
The team can attend communication and leadership courses, including on managing challenging conversation and working in a multigenerational environment. (Firm F)
Training on topics like behavioural biases, investigative interviewing, and assessing the reliability of memory. (Firm D)
Internal forensic team provides training on behavioural biases. (Firms C and D)
Global guidance on principles for interviews, explaining the importance of each, with examples of good and bad questions, and a sample interview outline. (Firm A)
Intensive two-day external course on behavioural psychology with classes, workshops and example tools. (Firm E)
4.3 Use of specialists
Firms may use specialists who can support their RCA programme by contributing to the design of their RCA methodology and materials, training the RCA team and, in a few instances, directly assisting with RCA procedures. Below are examples of how firms are using specialists to support their RCA teams:
Diagram showing how specialists assist in different stages of the RCA programme. Outer circles represent areas of assistance, central area indicates general support: * Designing the RCA approach and materials (all firms) * Assist in training the RCA team (all firms) * Performing RCA (two firms)
| Training | Developing tools | Review of process | Performing RCA |
|---|---|---|---|
| * Bespoke external sessions on barriers/facilitators for bias and psychological safety. | * Behavioural specialist developing a bank of interview questions and prompts. | * Specialist producing a report on barriers to quality and potential improvements. | * Internal culture specialists reviewing and challenging the identification of causes and behavioural factors, and reviewing reporting. |
| * Interview and trust techniques. | * Behavioural specialists undertaking a review of the RCA process and taxonomy. | * Overall review of RCA process. | * Legal counsel supporting RCA for most significant findings based on investigative experience. |
| * Bimonthly discussions for RCA senior managers with psychologists. | * Internal business psychologist reviewing tools and methodology. | * Behavioural specialists attending some RCA interviews. | |
| * Sessions by internal legal, forensics, culture and business psychologist specialists on biases, difficult conversations, psychological safety, building relationships and data analysis. | * Psychologists supporting the review of culture survey results and focus groups. |
4.4 Review and oversight
When reviewing the outputs of RCA, leadership should consider whether the conclusions drawn are appropriately supported and consistent with their wider knowledge. Where various teams perform RCA, oversight by leadership can support the consistency of this work and identify what additional review procedures may be appropriate for individuals/teams that are less used to performing RCA.
At one firm, the results of the RCA are subject to review by challenge panels made up of people outside of the RCA team. Objective challenge can help consider a broad range of insights and perspectives applied to the data analysis, and support an unbiased determination of the root causes.
4.5 Timing and deadlines
RCA for ISQM (UK) 1 deficiencies should be completed before the conclusion of the annual evaluation to inform the assessment of deficiencies. Initiating RCA promptly can also help ensure people's memories of events are fresh, reduce the likelihood of key people having left the firm, and make supporting information easier to access. Timely completion of RCA also means that remediating actions can be designed and implemented sooner so the firm can proactively manage the quality of its audits. However, this needs to be balanced with ensuring the individuals involved have sufficient time for reflection. For audit engagements, this could mean considering where it may be appropriate to begin planning and performing the RCA once the findings are known, before the inspection report is finalised.
| Firm A | Firm B | Firm C | Firm D | Firm E | Firm F | |
|---|---|---|---|---|---|---|
| Firmwide/SQM RCA timeline | No specific timeline | Within 60 days after the findings reported to the process owner | 60 days after the deficiency is identified (for FY24) | No specific timeline | No specific timeline | Within 6 weeks and complete before the start of global monitoring activities |
| Audit engagements RCA timeline | 3-6 weeks after the conclusion | 60 days post-conclusion for improvement required and 30 days for non-compliant | 60 days after conclusion | 6 weeks after reporting of the conclusion | 60 days after reporting (changing to 45 days) | 6 weeks after conclusion or close of appeal window (internal reviews only) |
At one firm, there were examples of active monitoring of emerging deficiencies enabling more timely RCA, with RCA being performed on initial compliance testing findings, before the testing programme was complete, so improvements could be implemented in the testing period.
4.6 Methodology
All firms use a broadly similar process to perform RCA, with variation in the inputs and methods at each stage. The diagram depicts this process and identifies several inputs used.
Flowchart illustrating the RCA process and its inputs:
- Inputs: AQIs, other data points; Focus groups; Interviews; Taxonomy; Significance of related findings.
- Process Flow:
- Identify and define finding for RCA
- Data gathering (with input from Working papers*)
- Speaking to relevant parties (with input from Questionnaires)
- Identify root causes (with input from Frameworks for weighting)
- Identify themes (with input from Record of recent and historic causes)
Most firms have extensive RCA methodology for all RCA on audit inspection findings.
Firms historically performed RCA for some firmwide findings and ethics breaches but are now increasing and/or formalising other RCA programmes due to ISQM (UK) 1. All firms have high-level guidance on firmwide RCA methodology but, given the diverse nature of firmwide matters, there is more emphasis on tailoring this methodology for different matters.
Example: At one firm, the RCA process for a firmwide deficiency relating to the completeness of training for new joiners, was planned by the core RCA team following preliminary analysis by the process owners and SQM team. The RCA team collected system data on new joiners and their training allocations, reviewed the data transfer between the HR and training systems, and interviewed the Compliance and IT teams. This led to identifying the system-based root causes and enhancing the actions proposed by the process owners.
Audit engagements RCA: At one firm, the extent of RCA procedures varies based on whether the findings were low, medium or high risk. This drives the number of hours allocated and extent of data collection, so the firm focuses time and resources on critical issues. At another firm, there are four levels of RCA for quality matters, covering good practice, adverse audit engagement findings, very significant adverse audit engagement findings, and thematic reviews or deficiencies in the SoQM. The levels drive the expected RCA tools and techniques, how and which stakeholders are involved in interviews/focus groups, and who performs and supports the RCA process.
Firmwide RCA: One firm is in the process of developing a firmwide RCA methodology, while another has already enhanced its guidance for firmwide RCA. Other firms have high-level guidance to be tailored based on the finding being analysed.
*For audit engagements
4.6.1 Data gathering
Analysis of quantitative and qualitative data can help identify causes, better define the quality issue, or pinpoint where analysis and interviews should focus. Firms should assess what data would be useful, the extent of data needed, and how to analyse it. Data can be gathered at various stages of the RCA process: at the beginning for scoping, before interviews, or after interviews to support or challenge the outcomes.
4.6.1.1 Use of AQIs and other data points - audit engagement findings
Most firms use AQIs or other data points to identify trends, patterns and correlations to help support and validate findings. Common AQIs used by the firms on the RCA relating to individual audits include:
| No. of hours | Milestones | Audit quality measures | Executive involvement |
|---|---|---|---|
| * EQCR hours | * Percentage of review notes cleared by milestone dates | * Time to archiving | * Percentage of hours by partners and directors |
| * Total audit hours | * Meeting planning milestones | * Internal inspection results | * Absolute and relative engagement leader hours |
| * Phasing of audit hours | * Timeliness of deliverables from the entity | * External review results | * Engagement leader portfolio review |
| * Extent of overtime |
Other AQIs used less frequently include:
- The quality track record of the engagement leader.
- Whether the engagement team is up to date on all mandatory training.
- Partner and EQCR service at the firm and on the engagement.
- The number of years the firm has audited the entity.
- Retrospective risk rating (a risk rating for each audit based on risk factors identified through the RCA process).
- The actual audit hours versus budget (as a % difference).
- The number of consultation tickets raised.
- Hybrid working considerations.
At one firm, 19 AQIs, including timesheet data, engagement leader hours, office locations, audit risk rating, and training and compliance data, were used in a thematic RCA on PYAs to corroborate or challenge insights from the interviews/focus groups. Similarly, at two firms, AQIs are used to structure interviews and challenge or corroborate insights. For example, if the manager was identified as new in grade or to the firm, the interview might explore how much training or coaching they had. Similarly, timesheet data is used to corroborate or probe further where individuals raised high workload as a causal factor.
Two firms have created RCA dashboards tracking AQIs and causal factors, to identify correlations, understand recurring issues and develop predictive indicators of poor audit quality. Data is considered cumulatively, with comparisons between periods. Similarly, another firm, after each RCA cycle, aggregated over 20 AQIs to look for trends linked to good and poor quality.
4.6.1.2 Use of AQIs and other data points – firmwide
For firmwide RCA, AQIs and data may also help identify how, where and why gaps arose in the firms' processes. Firmwide matters are more diverse, so RCA teams need to consider what data points and sources are relevant. Example of types of AQIs or data points used included:
- Engagement-level AQIs (see previous list).
- Training metrics, including compliance data, rates of uptake, assessment results and feedback.
- Culture survey data.
- Management information on resourcing.
- Timing of processes and controls.
Example: One firm's data analysis identified concerns regarding compliance with personal financial independence rules for certain grades. The firm performed RCA using survey results on self-reported reasons for non-compliance, complemented by discussions with the Ethics and Compliance teams.
4.6.1.3 Review of audit files and working papers
Inspecting underlying audit working papers typically strengthens RCA reviewers' understanding of the technical matters involved and the judgements exercised by the Audit team. It can also enable interviewers to corroborate or challenge points from the interviews, regarding the nature and quality of the audit procedures.
One firm always expects RCA reviewers to examine some working papers, with detailed guidance on what to review and an indication of which issues are worth raising in interviews. Another firm expects working papers to be reviewed for RCA for high and medium-rated findings. At three other firms, the decision to review working papers is based on the judgement of the RCA team. At the sixth firm, the RCA team conducts walkthroughs with the individuals who prepared the audit work where findings arose, to understand it in a collaborative setting, rather than directly reviewing the working papers.
Where working papers are not reviewed, the RCA team needs to ensure it has sufficient understanding of the underlying audit issues and the work and roles of the different engagement team members to perform effective RCA.
4.6.2 Speaking to relevant parties
Speaking to relevant parties is the most direct source of information about causal factors. The main mechanisms are interviews, focus groups and questionnaires. Firms should identify the individuals they want to engage with and the most effective mechanism for that engagement, considering how best to create an atmosphere of psychological safety and how to encourage interviewees to share their insights honestly and freely.
4.6.2.1 Interviews- audit engagement findings
Interviews are conducted in groups or individually to explore perceptions of the causal factors. Group interviews are often useful for comparison and consideration of common findings, as they encourage team discussion and create a discursive atmosphere. This is particularly true where senior employees encourage and role model speaking up. However, group interviews may deter more junior individuals from raising views contrary to, or critical of, senior colleagues. This is especially relevant for significant adverse findings, and RCA teams are sensitive to this. Interviewees are selected based on the scope of RCA procedures and who was involved in the underlying audit work. They may include senior and junior UK employees, offshore colleagues, component auditors, experts or specialists, EQCRs, hot reviewers and other technical consultants. Junior and offshore employees may have valuable insight and perspectives on why issues arose and the dynamics of the audit team. Experts, hot reviewers or technical consultants may also bring a wider view of the audit process and challenges.
Examples: * At one firm, the RCA team interviewed offshore team members who performed the underlying work and the UK employees who supervised. * them, as well as reviewing evidence of the communications between offshore/UK colleagues. * At another firm, for a severe IT audit finding, the IT specialists (preparer and reviewer) were interviewed alongside the core audit team. * At another firm, on a thematic RCA review across engagements with similar findings, the engagement leaders were interviewed together to discuss the extent of commonality of the causal factors.
4.6.2.2 Interviews- firmwide
The choice of interviewees will vary based on the nature of the finding and the range of people involved in the relevant processes/areas. Interviewers need to have sufficient knowledge of firm processes and be able to exercise broad judgement.
Examples: * At one firm, to understand a deficiency in the learning management system, the IT, compliance and learning teams were interviewed. This resulted in identifying more comprehensive remediating actions. * Another firm's RCA team interviews stakeholders and other teams that may have insight into firmwide processes (for example, members of panels for acceptance and continuance or risk assessment decisions). * At another firm, analysis of data, such as timesheets or details of ethics and compliance breaches, is used to select interviewees (for example, which individuals undertook training at certain times or had certain compliance issues).
4.6.2.3 Focus groups
Focus groups can be used to gather a wider range of views, on selected topics or themes, from individuals not directly involved in the finding, usually within a particular group or employee grade. They can be used to understand widespread issues, and to corroborate or challenge perspectives coming from specific interviews.
Examples: At one firm, a focus group was used to understand why junior colleagues were not using guidance. A group of employees were interviewed to understand how they currently use guidance and what barriers prevent them. They were also asked to locate common points of guidance, to observe whether, and how easily, they could. The results were used to corroborate identified root causes and design remediating actions.
Another firm, for an identified deficiency in director-led engagements, held focus groups with directors from across multiple business units, who had achieved positive and adverse quality results, to understand their challenges and experiences as engagement leaders and the factors that enable quality. A questionnaire was also used to gather data on their portfolios, their wider business roles and the composition of their teams.
4.6.2.4 Use of questionnaires (audit engagement findings and firmwide)
Firms can use questionnaires to gather information more consistently and to encourage people to reflect on their experiences and share concerns, as this can help identify where individual interviews may be beneficial. To achieve this purpose, firms should ensure employees feel safe enough to be open and honest in their responses.
Some firms issue template questionnaires to interviewees for audit engagement RCA to gather specific data, understand work culture and mindset, and prompt initial consideration of causal factors. Questionnaires use a mix of multiple-choice questions and open-ended questions, and ask individuals to rate statements. Examples of questions/statements include:
- I had a positive work-life balance during this engagement.
- I felt my work on this engagement was valued and recognised.
- I clearly understood my role and responsibilities on this engagement.
- I received suitable coaching and subsequent honest feedback about my work.
- What were the other projects you were involved in during that time? Any applicable deadlines that applied to other projects?
- How did the client's engagement (in particular, timeliness and quality of deliverables) impact the quality of the audit?
- What will you do differently this year?
- What was the attitude of management towards the audit and team?
- How accessible was the Responsible Individual (RI) during this engagement?
- What is your view of the overall skills and technical knowledge of those working for you on this engagement?
For firmwide RCA, firms did not have standardised questionnaires, but several firms have used tailored questionnaires for specific projects.
4.6.3. Identify root causes
Firms use taxonomies to provide a structured framework to consistently identify and record root causes. Historically, they used taxonomies for audit engagement RCA. Taxonomies may benefit from periodic review and update so the categories remain relevant and reflect changes in the firm's processes, systems and focus areas. Firms should have a robust process to map identified causal factors to its taxonomy.
Causal factors identified from RCA over positive events may also be categorised and tracked, to help identify significant causal factors driving quality. This can be done through a separate, positive taxonomy or by mapping positive causal factors to the main taxonomy. Where a separate taxonomy is used, it may be useful if the high-level categories mirror those of the main taxonomy, to support aggregation.
All firms have a taxonomy for root causes arising from audit engagement RCA. Two firms have SQM-level taxonomies, for firmwide RCA, although one is much shorter. Others are considering developing SQM-level taxonomies. One firm also has a parallel behavioural taxonomy to support understanding and monitoring of its culture. At another, the taxonomy cross references between categories of root causes to prompt consideration of additional systemic factors. Another firm, for audit engagements, has an annual process to analyse causes, to identify whether new categories need to be added to its taxonomy, via a manual overlay. This process also considers whether any new categories should be added to track common causes from firmwide RCA.
Firms have review processes in place for the mapping of causes to taxonomies. At two, where all RCA interviews are attended by at least two RCA members, the interviewers each identify which taxonomy codes are relevant, then compare and discuss the judgements exercised to counteract bias.
4.6.4. Identify themes
Firms identify themes from causal factors revealed during file reviews to pinpoint key concerns and where there may be underlying issues in the firm's SoQM. Identifying common themes between audit engagement and firmwide RCA root causes can also highlight systemic matters.
Most firms have a formal process for using their taxonomies to identify themes from audit engagement findings, but not for aggregating root causes identified in audit engagement and firmwide RCA. At one firm, the RCA team does aggregate audit engagement and SoQM root causes to identify themes and recurrent causes.
When designing a taxonomy, firms should consider how to balance sufficient granularity so that root causes can be meaningfully tracked, with sufficient aggregation so themes can emerge.
| Firm A | Firm B | Firm C | Firm D | Firm E | Firm F |
|---|---|---|---|---|---|
| 3 levels | 2 levels | 2 levels | 2 levels | None | 3 levels |
Five firms use multi-level taxonomies to balance granularity and aggregation. This means starting with broader categories that break down into more granular causes.
To determine themes, firms decide how to aggregate the root causes identified. They do this by assessing how frequently each root cause arose and by weighting causes, to give more prominence to those that are deemed to have the most significant impact on audit quality.
| Firm A | Firm B | Firm C | Firm D | Firm E | Firm F |
|---|---|---|---|---|---|
| 4 ranks | No ranking | No ranking | No ranking | 2 ranks | 2 ranks |
Firm A uses four rankings – primary, secondary, overarching and other non-causal factors – with a different numeric weighting applied to each. Primary and secondary factors refer to more and less significant direct causal factors. Systemic factors with an indirect causal impact (e.g., insufficient cumulative Audit team experience), are classed as overarching and have a lower weighting. Factors are considered non-causal where they did not drive the quality result. This enables them to be tracked without being included in themes.
Firms also judgmentally weight causes based on the significance and nature of the related findings (e.g., key, non-key, or good practice), recurrence, when the findings arose and if remediation has begun. For example, one firm, without a formal ranking, has a two-stage process where judgmental adjustments can be made by the RCA team and then by an independent challenge panel.
5. Reporting to leadership
Periodic reporting of RCA results to senior leadership increases awareness among leaders and can help ensure RCA teams have access to sufficient buy-in from senior leadership when they encounter obstacles and challenges.
Those with operational and ultimate responsibility for the SoQM need to understand the results and themes from RCA, to inform their perspective on, and evaluation of, the SoQM. Reporting that considers results of multiple types of RCA may help leaders recognise common themes and identify where further actions are needed. Firms' reports vary in frequency, content and detail.
Four firms specifically report on RCA results to those with ultimate responsibility for the SoQM, and another firm is starting to do it. At the remaining firm, reports are sent to its Audit executives. The reporting frequency varies between firms, with one firm having recently started quarterly reporting, and another establishing a triannual process.
Two firms combine audit engagement and firmwide RCA results in a single report, with one including results for all RCA relating to corporate reporting, ISQM (UK) 1, and ethics/conduct matters relating to specific engagements.
Common matters covered in reporting includes:
- Recurring RCA themes.
- Aggregated RCA themes.
- Root causes linked with good practice points.
- RCA approach and process.
- Proposed improvements to the RCA process.
- Remediating actions planned.
5.1 Reporting to the audit practice to share learnings.
Sharing RCA results with the Audit practice, in a timely manner, can help make individuals more aware of how to achieve good quality outcomes, and how to avoid pitfalls. Where firms use offshore colleagues on audits, they are likely to benefit if the communications are shared with them too. Based on the nature of the underlying findings, it may also be relevant to share communications with other relevant function areas such as IT or HR.
Examples of mechanisms used by the firms include:
- Regular webcasts for monthly or quarterly updates and ad hoc webcasts specifically for RCA results.
- Annual training events and courses.
- Weekly digests and newsletters.
- Direct communication with individuals involved in the RCA process.
- Reporting to learning and methodology leads to develop training and intellectual resources.
There should be a balance between communicating what has been seen to go wrong and what can be done to achieve quality. This may be an opportunity to share case studies and storytelling.
6. Monitoring the effectiveness of RCA
Firms should assess the effectiveness of their RCA programmes, including where enhancements are needed, and the impact of any changes made.
6.1 Monitoring the effectiveness of remediating actions and recurrence of findings and causal factors
Assessing the effectiveness of remediating actions can show whether root causes were appropriately identified and understood, and if the RCA is successfully supporting continuous improvement. Where actions were ineffective, firms should consider whether this was due to the incorrect root causes being identified, or remedial actions not being appropriately designed and/or implemented. Effectiveness can be assessed by monitoring recurrence of quality findings and/or causal factors, and by targeted monitoring of the impact of specific actions, for more timely information.
6.2 Setting and monitoring key performance indicators (KPIs)
Firms can use KPIs to assess how efficient and effective their RCA processes are.
Examples of KPIs used by firms include:
- Number and type of RCA projects completed.
- Number of RCA projects completed in the target timeframe.
- Time taken to complete RCA.
- Progress on designing and implementing remediating actions.
- Timely completion of planned remedial actions.
- Number of recurring findings.
- Percentage of remediating actions deemed effective.
One firm has set numerous RCA KPIs for bi-annual reporting to Audit leadership. These include quantitative and qualitative measures. Each KPI is scored from one to five based on defined thresholds for that KPI. For KPIs falling below three, actions are planned to improve the process.
6.3 Gathering feedback from relevant stakeholders
Feedback can help assess the effectiveness of RCA programmes. Relevant contributors may include members of the RCA team, impacted business process owners and interviewees. Feedback may cover stakeholders' experiences of the RCA process, as well as their views on its rigour and the robustness of outputs. Several firms are in the process of strengthening these feedback mechanisms.
At one firm, a post-interview questionnaire has been developed to gather timely feedback on interviewees' experiences and the extent to which they felt supported in expressing their views. At three firms, the RCA leader performs regular self-assessments to understand the effectiveness of the RCA process and the sufficiency of resources, and to see whether any issues or challenges are emerging.
6.4 Periodic holistic evaluation of RCA programmes
Periodic evaluation of the holistic process, by senior leaders, can help identify areas for improvement.
At one firm, a central quality team reviews the effectiveness of the RCA process, including its scope, timeliness, resourcing and the quality of reporting. At another, the next RCA programme is to be reviewed by Internal Audit.
Financial Reporting Council 8th Floor 125 London Wall London EC2Y 5AS +44 (0)20 7492 230 www.frc.org.uk
Follow us on Linked in. or X @FRCnews