The content on this page has been converted from PDF to HTML format using an artificial intelligence (AI) tool as part of our ongoing efforts to improve accessibility and usability of our publications. Note:

No human verification has been conducted of the converted content.
While we strive for accuracy errors or omissions may exist.
This content is provided for informational purposes only and should not be relied upon as a definitive or authoritative source.
For the official and verified version of the publication, refer to the original PDF document.

If you identify any inaccuracies or have concerns about the content, please contact us at [email protected].

IFRS 9 Banking Audit Methodology Application

The FRC does not accept any liability to any party for any loss, damage or costs however arising, whether directly or indirectly, whether in contract, tort or otherwise from action or decision taken (or not taken) as a result of any person relying on or otherwise using this document or arising from any omission from it.

The Financial Reporting Council Limited is a company limited by guarantee. Registered in England number 2486368. Registered Office: 8th Floor, 125 London Wall, London EC2Y 5AS.

Purpose and Scope

The FRC has undertaken a thematic review of the Big 4 audit firms' methodology and guidance around IFRS 9 - Financial Instruments and assessed the quality of individual audit teams' application of these resources. The purpose of this thematic is to promote continuous improvement in the quality of the audit work performed over the accounting and disclosure of financial instruments, with a focus on the audit of Expected Credit Losses (ECL) for larger banks, by providing auditors and audit committees with examples of good audit practice and common audit challenges.

Auditing financial instrument balances accounted for under IFRS 9 is complex and requires extensive audit consideration, with significant interrelation between the different aspects of audit work performed. Within this report, we have focused on the following nine elements:

Classification and Measurement (C&M) of financial instruments
ECL modelling
ECL model assumptions
Measuring Significant Increase in Credit Risk (SICR)
Modelled Multiple Economic Scenarios (MES)
Post Model Adjustments (PMAs)
Completeness and accuracy of data inputs
Individually assessed exposures
IFRS 9 disclosures

We performed the review by benchmarking key aspects of each firm's methodology and assessing a sample of working papers for a recently completed audit by each firm to consider how the methodology was applied in practice. Our report is based on this work, plus additional observations taken from bank audits inspected by our AQR team during the period in which IFRS 9 has been effective. Specifically, we reviewed:

The Big 4 audit firms' IFRS 9 methodology and guidance used to audit financial statements for the year ended 31 December 2021; and
Specific audit working papers from four individual audits across the nine areas noted above. These four audits, one from each of the Big 4 audit firms, represented two FTSE 100 banks and two other listed banks, all with reporting periods of 31 December 2021. These audits were not selected as part of the cycle of the 2021/22 AQR inspections.

We focused this thematic on the audit methodology over ECL determined using the general method, which is required to be used by banks for their loan portfolio measured at amortised cost and fair value through other comprehensive income. We have not reviewed audit methodology specifically over the simplified ECL method (typically used by entities for trade receivables, contract assets and lease receivables). The simplified method allows ECL to be measured without the need for certain judgements (for example, not needing to evaluate SICR, as all ECLs are assessed on a lifetime basis). However, certain credit risk assumptions, and incorporation of forward-looking macroeconomic information, are common across both methods. The IFRS 9 audit methodology is expected to be tailored and applied for each individual audit, to ensure that similar judgements are consistently evaluated, irrespective of the method used by management.

Key Observations

Since the implementation of IFRS 9 in 2018, each of the Big 4 audit firms have developed comprehensive IFRS 9 audit methodology, designed to help audit teams perform consistently high-quality audit work, particularly through the involvement of dedicated specialists and experts (such as credit modellers and economists). This is an example of the firms' continuing investment to improve the quality of their audit work and to respond to the variation in audit quality observed through our AQR inspections.

Audit methodology

Our review found some strong methodology in the areas of auditing ECL models and assumptions, including developing guidance on how audit teams should design their model risk assessment and testing approach. We have also seen comprehensive templates to enhance the quality and consistency of teams' testing C&M of financial instruments, as well as when testing individually assessed exposures. However, we have identified some areas for specific improvement, such as the guidance provided to teams on testing of SICR, which, for example, could be enhanced to provide more specific information on how to test the more judgemental elements of SICR.

We also noted that, in most cases, the firms' methodologies did not establish expected 'minimum audit procedures' for IFRS 9-related balances. While audit approaches must be tailored to individual audits, providing guidance on minimum procedures can offer a strong foundation upon which audit teams can develop an efficient and effective audit approach. Furthermore, establishing minimum audit procedures allows a firm's cumulative knowledge and experience to drive consistency within audit approaches, leading to higher-quality audits.

Application of methodology

We have seen examples of effective application of the firms' templates and tools resulting in high-quality audit work. This was particularly in:

the use of templates for individually assessed exposures;
automated tools to perform tests across all loans within a portfolio; and
the evaluation of the ECL models through independent recoding and recalculations by the firms' experts.

We have also seen instances of good practice and strong challenge of management where audit teams have used an independent challenger model (developed by the audit team to challenge the bank's calculations) to test a complex PMA. Another team exercised a high degree of professional scepticism leading to additional PMAs being recognised to address limitations identified in the bank's MES approach.

Despite the good practices noted above, we continue to observe instances of poorer application of methodology by some audit teams across a number of aspects of auditing IFRS 9, particularly ECL modelling, SICR, MES and individually assessed exposures. Examples of poorer application included audit teams de-scoping certain ECL models from their testing approach without sufficient supporting evidence, or not sufficiently considering the risk of management making manual overrides to their SICR criteria and thereby materially adjusting the stage allocations of exposures.

Common good practice characteristics

Through this thematic (focused on the Big 4 audit firms), and in our recent AQR inspections of this area (covering audit firms beyond the Big 4), we have observed audit teams typically performing higher-quality audits of IFRS 9-related balances when the following common features were present:

The audit team engaged the firm's internal credit modelling specialists and economists to evaluate and challenge management around all critical assumptions and judgements which materially influenced the ECL estimates, and specifically considered the appropriateness of plausible alternative assumptions.
A granular risk assessment was performed, identifying and assessing each constituent part of the ECL calculation and evaluating the impact of calculation methods, data inputs, model assumptions, and PMAs to determine which matters were of higher audit risk.
An evolving audit approach that was reassessed throughout the audit, which considered and tested the design and operation of the bank's controls around critical ECL elements. This ensured sufficient assurance was obtained as updated risk factors were identified during the course of the audit (for example, designing and performing additional substantive tests when management's monitoring controls were found not to be operating as initially planned).

While the key concepts of auditing financial instruments accounted for under IFRS 9 are now more established, audit firms should ensure that their methodology remains appropriate and responsive to new factors impacting on credit risk (for example, new idiosyncratic risks such as COVID-19 or Environmental, Social and Governance (ESG) related risk factors). This includes fostering strong communication both across audit teams and externally with audit committees.

Audit Methodology and Application Observations

In this section, we outline the findings of our review of audit methodology and its application across the nine key areas of IFRS 9 as outlined above, by:

Summarising each area and its importance to high-quality auditing;
For context, providing a brief overview of some procedures audit teams would typically perform in each area;
Describing good practice identified from audit firms' methodology and ancillary guidance;
Outlining good practice identified through the application of the audit methodology on the four audits reviewed; and
Providing examples of poorer audit methodology or application, as potential pitfalls for others to avoid on future audits.

While these findings are primarily based on the work performed specifically for this thematic, where helpful, we have added additional observations from recent AQR inspection work.

1. Classification and Measurement of Financial Instruments

Why is it important?

The C&M requirements of IFRS 9 determine whether an entity measures a financial instrument at fair value (either through profit and loss or other comprehensive income) or at amortised cost. This determination is important, as it can have a significant impact on the valuation of an entity's balance sheet and financial results.

What does this mean for auditors?

The classification of financial assets requires judgement to be applied. The two tests that comprise the C&M assessment for financial assets are the purpose for which a financial asset is being held (Business Model (BM) test) and whether an asset's contractual cash flows represent Solely Payment of Principal and Interest (SPPI). Audit teams need to test both the appropriateness of management's BM assessments and SPPI considerations. Typically, this would include:

Understanding and challenging the various BM designations given the facts and circumstances of the entity, as well as consideration of management's objective for a particular class of assets, for example, obtaining evidence that a class of instruments is indeed held for the purposes of collecting cash flows rather than being held to sell.
For SPPI, assessing whether the asset's contractual cash flows represent solely payments of principal and interest. For homogenous financial assets, this can be done collectively, but care needs to be taken where assets are bespoke, or have been reclassified or modified, noting reclassifications are not expected to be common in practice.

Good practice identified in firm level methodology and guidance

Requiring audit teams to perform an annual BM assessment for all portfolios, so that any changes to portfolios are considered (rather than relying on assessments rolled forward from previous periods).
Guidance to audit teams on when to engage the firm's financial instruments specialists to assess complex features for SPPI testing (for example, requiring their use for certain types of entities or where a significant risk in relation to IFRS 9 has been identified).
Providing BM and SPPI checklists/templates to guide teams through their compliance testing, thereby driving enhanced audit consistency.
Implementing Automated Tools and Techniques (ATTs)¹ which use International Securities Identification Numbers (ISINs) and Committee on Uniform Securities Identification Procedures (CUSIP) numbers to facilitate large scale C&M testing of registered investment securities.

Good practice identified on individual audits

Testing full populations of financial instruments using ATTs, with clear evaluation by the audit team (assisted by accounting specialists) of all exceptions identified.
Consideration of whether specific characteristics of a financial instrument result in it failing the SPPI test (for example, where loans have variable interest rates linked to specific ESG targets) and how these terms impact on the SPPI conclusion.

In recent AQR inspections, we have also identified instances of high-quality procedures over SPPI testing, where detailed evidence supported how audit teams evaluated the different types of assets to ensure tailored SPPI assessments were performed across each type of asset.

Examples of poorer audit methodology or application

Methodology

Lack of guidance issued to audit teams on the extent of follow up required for exceptions identified when using ATTs for SPPI testing, before concluding that exceptions are not significant.

Application

Obtaining limited corroborative evidence to support the underlying C&M testing, with an over-reliance on management representations (for example, accepting representations that assets had similar characteristics without evidence or accepting that no non-standardised terms exist without sufficient testing).

In recent AQR inspections, we have also identified instances where there was insufficient evidence supporting some audit teams' independent model code review, particularly where audit teams did not assess whether the code implemented actually followed the bank's IFRS 9 methodology.

2. Expected Credit Losses – Modelling

Why is it important?

IFRS 9 does not prescribe any specific modelling techniques, resulting in significant variation in how banks have constructed their modelling infrastructure. Often banks will not just have one ECL model, but a modelling landscape of a number of models that work together to compute the final ECL estimate. For example, banks often have separate Probability of Default (PD), Loss Given Default (LGD) and Exposure at Default (EAD) models, the results of which are used to calculate the ECL.

What does this mean for auditors?

Auditors need to assess whether the ECL model design, methodology and calculation are compliant with IFRS 9 requirements, and that the data used in the models is complete and accurate. Where banks have a multitude of models, the auditor needs to risk assess each model to conclude on the extent of testing required for each. Typical procedures may include:

Evaluating management's model governance process, including model monitoring and validation controls.
Reviewing the conceptual soundness of models and their compliance with IFRS 9.
Testing to assess whether models are operating as designed. This could be done in a variety of ways, including model code review, model rebuild or potentially using a challenger model.
Reperforming management's model validation and monitoring controls, where controls reliance is possible.
Identifying model weaknesses and limitations and assessing the sufficiency of their remediation.
Testing of the underlying data (including inputs and assumptions) and mathematical accuracy of the ECL models.

Good practice identified in firm level methodology and guidance

Clear guidance on the use of credit modelling specialists, including when they should be engaged and their specific roles and responsibilities around each element of ECL modelling.
Guidance to help teams to design a model testing audit approach commensurate with their risk assessment and providing clear direction on required testing before conclusions can be formed.
Reminders to teams to, where applicable, re-perform key monitoring tests, when evaluating model monitoring controls.
Guidance and supporting templates to assist teams in performing appropriate risk assessments and retaining sufficient quantitative and qualitative evidence to support their judgements.
Ancillary information provided to audit teams requesting significant risk assessment designations to be as focused as possible (for example, focused to a specific assumption for a certain portfolio).
Providing descriptions of what procedures are expected to be performed and assurance obtained when a team opts to recode a model, rebuild (partially or fully) a model, or create a challenger model.

Good practice identified on individual audits

Risk assessment of all individual ECL models, underpinned by qualitative and quantitative evidence to support risk conclusions reached (for example, quantification of the impact of individual model components by portfolio or sub-portfolios).
Reperformance of the model validation and monitoring controls, supported by detailed evidence by the audit and credit modelling teams to demonstrate how the audit team's model conclusions reconciled or differed from management's conclusions.
Significant proportion of models being tested through re-coding (based on the bank's methodology), or independently re-building elements of models, with assurance obtained over both the mechanical accuracy of calculations, as well as testing that models were operating as designed.
Where different inputs, methods and assumptions were used by the credit modelling specialists compared to those used by management, these were clearly justified and, where the testing highlighted differences, detailed evaluations were performed to conclude on their impact.

In recent AQR inspections, we observed audit teams performing a comprehensive assessment of the total aggregated impact of the combination of specific modelling differences and all other differences identified in other areas of the IFRS 9 audit.

Examples of poorer audit methodology or application

Methodology

Limited controls testing guidance despite the inherent complexity and significance of these controls, including a lack of examples of how to test typical model controls (for example, model development, model monitoring and model validation controls).

Application

Limited quantitative evidence to support the basis for de-scoping certain sub-sets of models.
Limited evidence retained to support the model validation and monitoring controls re-performance testing.

In recent AQR inspections, we have identified instances where there was insufficient evidence supporting some audit teams' independent model code review, particularly where audit teams did not assess whether the code implemented actually followed the bank's IFRS 9 methodology.

3. Expected Credit Losses – Model Assumptions

Why is it important?

ECL provisions and the models used to calculate them are based on underlying assumptions of credit risks within the portfolio – typically PD, LGD and EAD assumptions. These assumptions are often highly material to the ECL calculation and can be both complex and at risk of management bias and estimation uncertainty.

What does this mean for auditors?

Auditors should critically challenge management's model assumptions, such as PD assumptions, and consider all relevant risks around the development and application of these, including:

Assessing key assumptions, often with the help of credit modelling specialists. This would include evaluating the reasonableness of the assumptions and their compliance with IFRS 9, including looking at how they were developed, what data they are based on (both historical and forward-looking), and whether they remain appropriate for use at the reporting period.
Testing the appropriateness of other assumptions used in the ECL model calculations where these have a material impact, including contractual and behavioural life assumptions or Effective Interest Rate (EIR) proxies.

Good practice identified in firm level methodology and guidance

Clear templates for audit teams on how to test key model assumptions (for example, standalone templates which can be tailored and applied to varying management assumptions, which facilitates consistent testing of assumptions).
Reminder for audit teams to review all assumptions individually and in aggregate at the risk assessment stage, with additional guidance on the qualitative and quantitative evidence to be retained when concluding assumptions are not significant to the ECL calculation.
An automated tool that helps the audit team integrate their risk assessment into all stages of the audit and update as needed. When the tool is used on model assumptions, this facilitates the identification of all material risks for each portfolio with ECL provisions and allows clear links to procedures executed and conclusions drawn in response to those risks.

Good practice identified on individual audits

Effective and timely use of specialists, panels and consultations evidenced through high-quality memoranda and other templates that clearly showed the specialists' approach and results of their testing or consideration of management's assumptions.
Retaining comprehensive qualitative and quantitative evidence to support risk assessment conclusions made for each of the model assumptions, including evidence to demonstrate that the assumptions had been reassessed throughout the audit.

Examples of poorer audit methodology or application

Methodology

Inconsistency in guidance where topics were covered in multiple areas.

Application

Limited evidence supporting risk assessment conclusions reached, particularly where assumptions were concluded as not being material to the ECL calculation.
Insufficient integration of the credit modelling team's procedures into the audit testing, which led to poor audit trails to demonstrate all material ECL model assumptions were challenged and evaluated before audit conclusions were reached.
In instances where numerous assumptions were deemed to be immaterial, insufficient evidence to demonstrate why the cumulative impact was also immaterial.

4. Measuring Significant Increase in Credit Risk

Why is it important?

At each reporting date, banks need to use reasonable, relevant and supportable information to assess whether the credit risk of a financial asset held at amortised cost has increased significantly since initial recognition. If credit risk has not increased significantly since initial recognition, a 12-month ECL (Stage 1) is recognised, and if it has increased significantly, a lifetime ECL (Stage 2) is recognised, which may be significantly higher than a 12-month ECL.

What does this mean for auditors?

IFRS 9 allows banks to select their own criteria for identifying SICR which is relevant to the credit risk factors present in their portfolios, as well as considering rebuttable presumptions around SICR. This results in significant judgement being applied and requires careful consideration by the auditor, typically through procedures such as:

Assessing the appropriateness of the bank's SICR criteria to check whether these reflect how credit losses might occur within that portfolio.
Understanding and testing the entity's governance over SICR criteria and the monitoring of changes to ensure that the SICR criteria remain up-to-date and effectively measure risk factors.
Testing whether the criteria are operating as designed. For example, where credit scores are used as a SICR criteria, ensuring that when changes to scores hit certain thresholds, the exposures are moved between stages appropriately.

Good practice identified in firm level methodology and guidance

Practical guidance to assist teams to challenge whether management's SICR criteria are sufficiently forward-looking to capture changes which would impact current credit risk, for example, evaluating whether future legislative changes may impact on the credit risk of counterparties.
Outlining comprehensive expectations of the work to be performed:
- On judgemental qualitative criteria, which are much more subjective in nature than quantitative criteria and therefore harder to test; and
- When testing the staging of complex corporate exposures, such as project financing, which is often dependent on the revenues generated from a single project for both the source of repayment and as security for the exposure.

Good practice identified on individual audits

Entity specific risk assessment of the SICR process, which demonstrated a thorough understanding and assessment of the criteria, the mechanics of their operation and their relative impact on the ECL calculation.
Testing SICR criteria across a retail and wholesale portfolio to verify that the SICR criteria were operating as designed at year-end, including re-calculating a sample of SICR criteria from source data. For example, re-calculating the arrears status of loans and ensuring those exceeding the determined SICR thresholds had been moved into the correct stage.
Use of ATTs to re-perform the SICR staging for 100% of loans to test that the SICR criteria were correctly implemented and applied accurately to the year-end loans.

SICR is an area that we review as part of each AQR inspection of bank audits, and in recent inspections we have seen examples of good practice including:

Timely engagement of specialists/experts such as credit modellers, economists and the accounting technical team to inform the SICR risk assessment.
Portfolio level assessment of SICR methodology for both quantitative and qualitative criteria to evaluate their compliance with IFRS 9, with effective use of publicly available guidance (for example, banking regulatory guidance) and consideration of leading and lagging indicators, as well as the level of reliance placed on the backstop criteria required by IFRS 9.

Examples of poorer audit methodology or application

Methodology

Lack of specific guidance for audit teams around the following judgemental elements of SICR:
- Certain qualitative and/or highly judgemental criteria, and
- Management overrides/adjustments to the SICR criteria.
Lack of clear guidance as to when audit teams should consider using ATTs instead of a manual substantive testing approach.

Application

Insufficient consideration of the risk of management override to the stage allocation, including the risk of overriding underlying data driving SICR criteria, as well as the risk of manual transfers between stages.

In recent AQR inspections we have seen insufficient testing over the completeness and accuracy of all relevant SICR criteria, with audit teams only testing selected criteria without justifying why no testing is required over others.

5. Modelled Multiple Economic Scenarios

Why is it important?

IFRS 9 requires ECL be measured in a way that reflects an unbiased and probability-weighted amount that is determined by evaluating a range of possible outcomes. This involves considering forecasts of future economic conditions to evaluate the expected performance of loans, and the related credit risks, to calculate the level of expected losses. This assessment is judgemental and subject to a number of assumptions and estimates.

In practice, this will involve the generation of MES, based on forecasts of underlying macroeconomic variables impacting on credit risk. These are incorporated into the ECL models and reflected in the calculated provision. This will also often use complex models to reflect the non-linearity of ECL movements to changes in underlying macroeconomic variables, which may have their own limitations (for example, using Monte Carlo models can result in a narrower range of scenarios compared to more discrete methods).

What does this mean for auditors?

Auditors should evaluate the basis of management's MES, including which macroeconomic variables are significant and relevant to credit risk, how scenarios are constructed, and how the relative probability of those scenarios occurring is determined. This would typically involve:

Understanding the bank's methodology and approach to utilising historical and forward-looking macroeconomic information, including an evaluation of the impact of non-linearity.
Assessing the completeness and suitability of the macroeconomic variables used and evaluating the appropriateness of the scenarios generated and the reasonableness of the weightings applied.
Engaging specialists or experts as needed to review, assess and conclude on the appropriateness of management's determined methodology and assumptions, its compliance with IFRS 9 principles, and to test the application of management's methodology in the ECL models.

Good practice identified in firm level methodology and guidance

Rebuttable presumption that economists will be engaged to test the incorporation of forward-looking information, given the specialised knowledge often needed to assess the reasonableness of forward-looking macroeconomic forecasts.
Templates which guide audit teams to assess critically the work performed by the audit firm's credit modelling specialists and economists and ensure exceptions are sufficiently followed up.
Guidance for audit teams on the use of the economists' work, including an expectation that audit teams should first evaluate management's MES methodology. Where there are unresolved differences in opinion with management (for example, regarding the approach to measuring the impact of certain forward-looking assumptions), the guidance notes audit teams should then explicitly consider the need to develop independent economic scenarios, and reasonable ranges, based on the firm's own reasonable economic assumptions. Where independent scenarios are developed, the guidance outlines the type of evidence which would be required to support the high and low end of their independent ranges.
Guidance that under certain circumstances (for example, where modelling risk is assessed as high) challenger models (being models created independently by the firm using their own assumptions and scenarios) could be used for aspects of the modelled MES testing.
Providing audit teams with a Frequently Asked Questions list on testing MES, which provided details of the types of audit challenges which could be raised given differing MES approaches.
Encouraging the use of in-house ATTs to, for example, generate future economic forecasts for individual economic variables based on historical actuals using multiple data points; and benchmark the reasonableness of the bank's IFRS 9 macroeconomic assumptions.

Good practice identified on individual audits

Evidence of effective challenge of the MES approach by the audit team and their experts, resulting in an additional PMA being recognised by management. This PMA was audited by the firm's credit modelling specialists, with additional factual and judgemental differences being identified which were specifically evaluated. The audit team supported its conclusions over the appropriateness of this PMA, and how this better reflected the impact of MES on the bank's portfolios, with comprehensive evidence linked back to the challenges raised by their experts, and also produced high-quality reporting to the audit committee.
Assessment of each of management's modelled MES assumptions, which included detailed explanations and supporting evidence for the expected ranges which were independently developed by the audit team-engaged economists.
Effective engagement of applicable specialists/experts on the audit with clear evidence within their reporting templates demonstrating the extent of their procedures and the audit team's evaluation of this work.
Specialists and experts performing independent procedures over the base and alternate scenarios and challenging management's probability weightings based on their own assessment of reasonable base and alternative scenarios.
Use of a number of external sources to corroborate and challenge the appropriateness of the underlying economic variables used within management's base forecasts.
Clear and comprehensive testing by both the audit and economist teams over the completeness and accuracy of macroeconomic variables used.

Examples of poorer audit methodology or application

Methodology

Limited guidance on when to engage the economist team and only requiring them to be engaged when a significant risk has been identified for modelled MES.

Application

Limited evidence of the understanding and testing of the non-linearity of variables, for example, understanding why property values might be decreasing by 5%, but ECL only increasing by 1%.
Not testing the completeness of macroeconomic variables that would materially impact the credit risk within each portfolio.

In recent AQR inspections we have raised findings with regards to the testing of forward-looking information, including:

Lack of testing the completeness of macroeconomic variables used by management;
The sufficiency of procedures performed by the economist over the base and alternate scenarios and challenging management's probability weightings; and
The extent of testing needed to assess any potential double counting of forward-looking factors in the ECL calculation (for example, considering whether forward-looking factors being included within the ECL calculation through both MES and PMAs).

6. Post Model Adjustments

Why is it important?

ECL models will typically have some known limitations (for example, limited historical default data for PDs) or simplifications to allow for the models to operate in practice (for example, credit assumptions not capturing risks arising from interest only mortgage redemptions) and will not be designed to capture all idiosyncratic events (such as the impact of COVID-19).

As such, management therefore needs to record additional adjustments, after the ECL models have performed their calculation, to ensure that these risks and limitations are reflected in the final ECL provision. These overlays to model output are typically referred to as PMAs and are usually highly subjective, and can contribute, on occasion, more to ECL charges than the ECL models themselves.

What does this mean for auditors?

Auditors need to assess the appropriateness and completeness of PMAs to ensure that they achieve their intended purpose, and that sufficient PMAs are in place for known issues. Typically, procedures will involve:

Obtaining the list of PMAs applied by management and gaining an understanding of why management determined that they were required.
Evaluating and challenging the appropriateness and reasonability of management's PMAs.
Challenging the inputs, methods and assumptions used for PMAs, which can be more challenging for the judgemental, complex, and other idiosyncratic macroeconomic PMAs, where teams might partially or fully re-calculate the PMA from source information.
Assessing the PMA list for completeness by considering whether all model limitations (for example, identified through model monitoring procedures) and other idiosyncratic events have been covered by a PMA.
Testing to assess whether PMAs have led to any double counting within the ECL calculation (for example, whether the event which triggered the need for the PMA would already be reflected in the forward-looking scenarios and weightings).

Good practice identified in firm level methodology and guidance

Providing clear resources to teams around:
- Assessing and understanding management's basis for applying PMAs including the idiosyncratic risks or model limitations being adjusted for;
- Evaluating whether PMAs are compliant with IFRS 9 and incorporate forward-looking information as at the reporting date;
- Testing potential double counting of risks that may already be captured by the core models; and
- Considering whether appropriate governance is in place over PMAs.
Reminding teams to link their PMA audit procedures with other ECL testing, in particular work over models, to ensure known data limitations and modelling issues have been appropriately addressed through PMAs.
Clear guidance as to when audit teams need to use specialists or experts, such as credit modellers and economists, in the audit of PMAs, with detail on the relevant roles and responsibilities of the audit team and the specialist/expert teams.

Good practice identified on individual audits

Clearly outlined risk assessment and tailored audit approaches, which separated PMAs into three distinct categories (data-related PMAs, other model limitation/risk-based PMAs, and other idiosyncratic macroeconomic PMAs).
Use of a challenger model to audit a highly judgemental PMA, where the challenger model was built by the audit firm's credit modelling team, using both their own, and other independently sourced, assumptions. The model was then applied to management's data and resulted in a differing quantified PMA, which was used to challenge management's PMA.
Re-calculating PMAs, either fully or partially, based on management's methodology, which provided strong evidence that the models used to calculate PMAs were operating as designed.
Effective use of an internal panel on a significant PMA, where a panel of senior executives (including members of the technical accounting team and other audit partners) provided their perspective on the PMA before concluding on its appropriateness.
Comprehensive PMA completeness assessment utilising the firm's internal specialists, which considered a number of factors, such as geopolitical risks, which was used to challenge management's PMAs.
Double counting considerations clearly evaluated and quantified with underlying evidence to support the audit and specialist teams' conclusions that these were not material to the ECL calculation.

Examples of poorer audit methodology or application

Methodology

Insufficiently differentiated guidance on the audit approach for modelling related PMAs compared to judgemental idiosyncratic macroeconomic PMAs, with limited examples to aid the audit team's testing.

Application

Limited evidence to support completeness of PMAs and how this reconciled to the outputs of model monitoring performed.
Lack of consideration around whether PMAs were double counting events which were already covered in the ECL models.

7. Completeness and Accuracy of Data Inputs

Why is it important?

The calculation of a bank's ECL allowance is dependent on complete, accurate and relevant data which is input into ECL models. Typically, banks' ECL models are part of a wider complex IT environment and may use dozens of data elements, from both internal and external sources. As well as driving the actual ECL model calculations, data is also used for model monitoring and validation controls, which are key to ensuring strong ECL model performance.

What does this mean for auditors?

To test the data used within the ECL calculation, auditors would usually:

Identify and understand all data flows feeding into the ECL models, which includes identifying all sources of data, including relevant internal IT systems (for example, data from the loan system), and external feeds (for example, credit bureaus).
Establish which data elements (for example, arrears status) from each data flow are being used in the ECL calculation and identify those which are material to the calculation, hereafter referred to as 'Key Data Elements' (KDEs).
Design audit procedures to test the completeness and accuracy of KDEs to appropriate sources.

ECL models are often run using 'model input datasets', which are aggregations of data from one or more systems or sources. Auditors need to consider the completeness and accuracy of such data sets, either through controls or substantive procedures, which could include the following typical audit tests:

Reconciling row counts or balances between source systems and the input data sets (completeness).
Testing KDEs for a sample of loans selected from the data sets back to source systems (accuracy).
Testing automated interfaces between IT systems and the ECL models or manual reconciliation controls.
Where KDEs are calculated automatically by IT systems (for example, balance in arrears), auditors typically test the design and operation of these automated controls.

Good practice identified in firm level methodology and guidance

Clear guidance to assist audit teams to identify a complete population of data elements, including guidance on involvement of internal specialists, for example, credit modelling specialists.
Practical guidance for audit teams on:
- The types of data, associated risks, and typical data sources used within ECL models (including both internally-generated and externally-sourced data); and
- The types of controls management may have in place in relation to the completeness and accuracy of data flows relevant to ECL (for example, manual reconciliation or automated interface controls).
Suggested testing procedures to guide teams as to which procedures might be appropriate to test KDEs in different specific circumstances (for example, when there are ineffective General IT Controls).

Good practice identified on individual audits

Good practice identified on individual audits has typically related to audit teams demonstrating a thorough understanding of the data used within the ECL models, including the relevant source systems and data flows, and the underlying IT environment. In identifying those data elements considered key for the ECL calculation (KDEs), specific good practice has included:

Developing a complete list of data elements used in the impairment calculations and evidencing how those data elements were used by the ECL models. Good examples have included inspection of the ECL models by the firms' internal credit modelling specialists to identify independently the data elements used by the models, with comparisons to supporting ECL methodologies and model documentation.
Audit teams considering which KDEs they would expect to see used by the ECL models, based on their walkthroughs or understanding of specific products, and verifying whether these have been included.
Comprehensive assessments of the KDEs, including justification as to why certain data elements were considered key, and others not.
Detailed testing over the KDEs, including details of source evidence supporting each KDE.

Examples of poorer audit methodology or application

Methodology

No reportable examples identified.

Application

Insufficient procedures performed to identify a complete population of KDEs used within the entity's ECL models (for example, only basing the assessment on inspection of management's ECL methodology without consideration of data elements noted from the audit team's product walkthroughs and data elements observed from the specialist's work on the ECL models).
Lack of justification for the exclusion of certain KDEs from testing.
Insufficient response to control deficiencies, in particular in relation to data transfer or reconciliation control issues, with the audit team not re-assessing the risks relating to data within the ECL calculation or considering the need for any further substantive or controls testing.

In recent AQR inspections, we have also identified issues relating to the actual testing performed over KDEs, including exceptions not being adequately assessed or followed up, and testing only focused on the accuracy of data and not completeness.

8. Individually Assessed Exposures

Why is it important?

The ECL allowance for larger or more complex exposures (for example, large corporate exposures) are often not calculated using a model, but are instead individually assessed so that unique factors, complexities and nuances can be specifically considered and incorporated into the ECL provision. Whilst this allows for a more tailored approach, it often increases the level of subjectivity and potential for management override.

What does this mean for auditors?

When auditors are testing individually assessed exposures, they need to evaluate the key judgements made by management, to assess whether these are reflective of the unique credit risks of an exposure. This will typically involve:

Identifying the bank's criteria for selecting loans that are individually assessed and testing the completeness of that population.
Selecting a sample of exposures for testing that responds to the nature and inherent risks associated with the portfolio (for example, using sector and size considerations), and selecting representative samples across other populations which are not tested through a risk-based sample.
Evaluating and challenging each of management's key assumptions underpinning the individual credit assessment, often with the help of specialists and experts (for example, to value more complex collateral).
Testing the probability-weighted MES applied to individual exposures.

Good practice identified in firm level methodology and guidance

Templates to assist audit teams perform individual assessments on a consistent basis and assist them in considering all IFRS 9 requirements. This includes prompts for teams to retain specific evidence over judgements as part of their evaluation, and to tailor these as relevant to the exposure being assessed. This facilitates review and quality control procedures, as samples are documented in a consistent manner across audits, and aids comparability.
Clear guidance to audit teams around when to engage specialists and experts to assist in reviewing individually assessed credit exposures (for example, when individual loans are supported by unique collateral).

Good practice identified on individual audits

Clear articulation for sampling rationale outlining the basis for the initial sample selection and the basis for extending samples to respond to any additional identified risks for the audit.
Obtaining independently sourced evidence to challenge management's judgements within the individual credit file reviews where these are material to the conclusions reached (for example, checking forecasted oil prices to external sources for loans where recovery is contingent on future revenue linked to global oil prices).
Consideration of unique impacts on exposures such as COVID-19 and ESG risks were executed to a high standard with specific evidence obtained to support how these risks impacted on the measurement of credit risk.

In recent AQR inspections, we have seen audit teams populating firm templates to a high standard, clearly demonstrating, with supporting evidence, how they have reached their conclusions on matters such as collateral valuations and discounted cash flows used by management.

Examples of poorer audit methodology or application

Methodology

Methodology, tools and templates that lack precision, specifically regarding the testing of forward-looking information within each credit file and challenge of MES assumptions.
Limited guidance or practical examples to facilitate assessment of more complex credit exposures (for example, when multiple recovery strategies are assumed which are conditional on future outcomes and the relative probability weightings and MES impacts are judgemental).

Application

Not verifying the completeness of the bank's individually assessed exposures population or ensuring the bank's processes have covered all loans which were expected to be individually assessed.
Limited evidence showing considerations of MES on individual exposures, including the methodology used and the probability weightings assigned by management, and limited evidence of work performed by specialists or experts in assessing the MES aspects of the credit exposures.

9. IFRS 9 Disclosures

Why is it important?

IFRS 9 triggered notable changes to requirements for financial instrument disclosures, which are outlined in IFRS 7 – Financial Instruments: Disclosures, including changes to existing requirements as well as bringing in significant new ones (for example, qualitative disclosures to demonstrate how the entity applied the IFRS 9 C&M requirements). Banks also need to consider the extent to which significant judgements and sources of estimation uncertainty need to be disclosed under IAS 1 – Presentation of Financial Statements, including the need to provide meaningful sensitivity analysis.

Disclosures are fundamental to allow for transparent and useful information to be communicated to users of financial statements, so that significant judgements and estimates which influence the ECL provision can be understood, evaluated and compared by the market.

What does this mean for auditors?

ISA 540 requires the audit of accounting estimates to include consideration of disclosures. Auditors should test the disclosures made by the bank to ensure this communicate management's the key judgements and estimates made by management, in an understandable manner to allow users to make informed decisions. This is achieved by evaluating and challenging an entity's IFRS 7 and IAS 1 disclosures for compliance with the standards, as well as assessing their completeness and accuracy.

Good practice identified in firm level methodology and guidance

Using IFRS 7-specific work programs which outlined disclosure requirements and expected testing thereof.
Internal hot and cold reviews of disclosures for a sample of banks and building societies.
High quality illustrative financial statements being prepared, which are relevant for banking/financial services entities and include reference to Disclosures about Expected Credit Losses (DECL) guidance.
Guidance to use specialists (for example, credit modellers, economists) to test and assess the adequacy of specific disclosures.
Reminders to communicate unresolved disclosure omissions to management and the Audit Committee, where these are not trivial.
Timely firmwide communications, training, practice alerts and updates relating to DECL good practice disclosures.

Good practice identified on individual audits

No reportable good practice identified.

Examples of poorer audit methodology or application

Methodology

Pre-issuance disclosure reviews only being performed on high-risk audits and other large non-listed financial institutions, and not considered for other entities which have complex or enhanced risks around disclosures.

Application

No reportable examples identified.

Appendix

Limitations of this thematic

We outline below the limitations of our scope as part of performing this thematic:

This thematic was primarily based on our review of the Big 4 firms' IFRS 9 audit methodology and ancillary guidance, and on representations made by the firms in response to specific questions. While instances of good practice have been highlighted, we do not provide any assurance over the sufficiency of an individual firm's methodology, policies and procedures.
For the four example audits, we did not have direct access to the full audit files and therefore did not assess the quality of the audits. The review of the example audits was based on a sample of key working papers, and other documents provided by the firm, and did not constitute an Audit Quality Review (AQR) inspection.
The good practice examples referenced in this thematic may not equate to good practice reported in the Audit Quality Inspection and Supervision Reports for individual firms, or good practice included in AQR inspection reports on individual audits. Similarly, examples of poorer audit work may not equate to issues which may be included in those reports.
We have not reviewed or inspected the firms' methodology or guidance in relation to General IT Control procedures or IT Application Control testing, as part of this thematic.
We have referred to the use of ATTs by the firms throughout this thematic, including instances where we consider this to represent good practice. The effective use of ATTs relies on such tools and techniques operating as intended, and being subject to the firms' regular internal accreditation/certification processes. The effectiveness of such processes have not been considered as part of this thematic.

Other relevant FRC publications

The following FRC publications should be read in conjunction with this thematic. These outline key auditing principles and messages that are also relevant for auditors to consider to achieve high-quality audits of IFRS 9-related balances:

What Makes a Good Audit
What Makes a Good Environment for Auditor Scepticism and Challenge
Professional Judgement Guidance
The Use of Technology in the Audit of Financial Statements
IFRS 9 Thematic Review: Review of Disclosures in the First Year of Application

Financial Reporting Council 8th Floor 125 London Wall London EC2Y 5AS +44 (0)20 7492 2300

www.frc.org.uk

ATTs: Automated Tools and Techniques – Technology used to perform risk assessment procedures and/or obtain audit evidence. ↩

File

Name IFRS 9 Banking Audit Methodology Application

Publication date 27 September 2023

Type Thematic review

Format PDF, 396.0 KB

Download original

IFRS 9 Banking Audit Methodology Application

File

Is this page useful?