Skip Ribbon Commands
Skip to main content
 
ARCHIVED CONTENT

The Utilisation of Internal Loss Data in the Measurement and Management of Operational Risk in Australian AMA Banks

Information Paper

Shane Wilson — July 2008

January 2013 - Update to this Information Paper

Please note that since this Information Paper was released in July 2008 there have been significant developments in operational risk management and measurement practices, and therefore the contents of this Paper should be considered in the light of these improvements. Authorised deposit-taking institutions, especially those with approval to use an Advanced Measurement Approach (as stated in paragraph 23 of APS 115), need to ensure where industry practices evolve and improve over time that these developments are assessed as part of their own practices. APRA will be releasing further guidance in relation to operational risk practices in the near future.

Copyright

© Commonwealth of Australia
This work is copyright. You may download, display, print and reproduce this material in unaltered form only (retaining this notice) for your personal, noncommercial use or use within your organisation. All other rights are reserved. Requests and inquiries concerning reproduction and rights should be addressed to:
Commonwealth Copyright Administration
Copyright Law Branch
Attorney-General's Department
Robert Garran Offices
National Circuit
Barton ACT 2600
Fax: (02) 6250 5989
or submitted via the copyright request form on the website http://www.ag.gov.au/cca

 

Disclaimer

While APRA endeavours to ensure the quality of this Publication, APRA does not accept any responsibility for the accuracy, completeness or currency of the material included in this Publication, and will not be liable for any loss or damage arising out of any use of, or reliance on, this Publication.

 

Inquiries

For more information on the contents of this publication contact:
Shane Wilson, Supervisory Support Division
Australian Prudential Regulation Authority
GPO Box 9836
Sydney NSW 2001
Tel: 61 62 9210 3000

 

Acknowledgements

Author: Shane Wilson
I would like to thank Harvey Crapp and André Levy for their helpful comments and suggestions.

 

 

Introduction

The Australian Prudential Regulation Authority (APRA) states in prudential standard APS 115 Capital Adequacy: Advanced Measurement Approach for Operational Risk (heretofore APS 115), that complex ADIs1 (heretofore banks) who wish to implement the Advanced Measurement Approach (AMA) for operational risk must make use of four data elements in the determination of their Operational Risk Regulatory Capital (ORRC). The four data elements, (Internal Loss Data (ILD), External Loss Data (ELD), Scenario Analysis (SA) and Business Environment and Internal Control Factors (BEICFs)) work in conjunction to enable a bank to transparently quantify their operational risk profile and determine commensurate regulatory capital.

As the data source which most closely resembles a bank’s internal risk profile, ILD plays a pivotal role in both the measurement and management of an institutions operational risk profile. Although mandatory data collection for operational risk has only recently come about as part of the AMA accreditation process, Medappa (2006) points out that “operational loss events may have been captured in some form in a variety of systems. However, these records would be incomplete with respect to the minimum requirements laid down by the Basel Committee for Banking Supervision (BCBS).” As such, the purpose of this paper is to outline the observed data collection practices of AMA accredited Australian banks and address some of the key challenges faced when collecting internal losses.

“The tracking of internal loss event data is an essential prerequisite to the development and functioning of a credible operational risk measurement [and management system]” (BCBS, June 2006), as it enables banks to tie their risk estimates to their actual loss experience. The scope and quality of the operational risk data collected by an AMA bank will have significant impact not only on the quantification of their risk profile, but also on their operational risk management decisions (BCBS, October 2006). For maximum benefit, banks should tailor the depth of their data collection to their risk management and measurement framework.

Banks need to assess whether the data will be used as a risk management and/or measurement tool and ensure that the depth of the data is fit for purpose. For example, the level of detail recorded for each loss will vary somewhat between banks using ILD explicitly in their operational risk quantification model and banks using ILD for validation or as a management benchmarking tool. That said, APRA has enforced minimum requirements with regard to the type of information that must be included in the loss capture database, i.e. “gross loss amounts, the date of the loss event and any recoveries, as well as descriptive information about the drivers or causes of the loss event” and indicators for credit and market related losses (APRA, paragraph 21, 2008).

In practice AMA accredited banks have generally captured an extensive range of information well above the minimum requirements set out by APRA, including indicators for near miss, strategic and reputational risk, internal and Basel business unit attribution and Basel event/risk type classifications. During the AMA accreditation process it was apparent that those banks who took a broad approach to the collection of operational risk losses were able to ensure that their Operational Risk Management Framework (ORMF)2 was able to adapt to changes in industry risk modelling practices due to the inbuilt flexibility of their data capture system.

Policies and Procedures

To ensure consistency in the collection and validation of operational risk loss data, banks must develop a set of policies and procedures that outline the rationale and process of collecting loss data. The policies should contain guidance and information about the collection of ILD, including:

  • Definition of operational risk
  • Roles and responsibilities
  • Capture and verification of operational risk loss data
  • Classification of credit, market and other risk related incidents
  • Date Recorded
  • Loss Amount
  • Currency

Definition of operational risk

All banks are able to implement a tailored definition of what constitutes an operational risk, for loss collection purposes. However, for the purpose of ORRC, all losses included in the AMA modelling process must conform to the definition of operational risk in APS 115.

Roles and responsibilities

A clear explanation of the roles and responsibilities of the group operational risk management function and business unit managers should be included in the banks operational risk policies. Each management function must be aware of their data management responsibilities to ensure the consistency and accuracy of the collected data.

Capture and verification of operational risk loss data

Extensive detail should be provided on the identification, collection, processing and approval of ILD. These procedures become a point of reference for any staff unfamiliar with the data collection process.

Classification of credit, market and other risk related incidents

APS 115 states that all credit and market related operational risk losses must be recorded in a bank’s loss capture database and flagged to ensure the losses are treated appropriately for regulatory capital. APS 115 is clear on the classification and treatment of credit and market related operational risk losses for regulatory capital. However, no official definition is given for other risk related losses because they are assessed under Pillar II. Banks must ensure that clear and compliant definitions of what constitutes a credit, market and other risk related loss (such as strategic and legal) are set out in the relevant policies to prevent regulatory capital arbitrage.

Date Recorded

The recorded date for a loss event may have significant impact on the assessment of the bank’s operational risk profile at a given point in time. Most AMA banks record more than one date in their internal loss database, but only use one of those dates for modelling purposes. The most commonly recorded dates include:

  • The date of occurrence
  • The discovery date
  • The recording date
  • The date of financial impact

Because some losses are frequently not detected until months, even years after the loss event has occurred, the date of occurrence and discovery date will often differ. However, due to increased staff awareness of operational risk, AMA banks are noticing a reduction in the amount of time between when a loss event occurs and when it is discovered. Banks who record and use the accounting or recording date for modelling purposes may find that their loss history becomes distorted if a number of losses are entered into the database on the same day. That being said, recording the accounting date for informational purposes allows for loss events to be easily reconciled with the General Ledger.

Because litigation losses may take several years before they are resolved, banks are careful not to assign a loss amount to an ongoing dispute, as this may increase the likelihood and size of the loss. As such, most banks record the accounting date or the settlement date for legal losses. Banks must ensure that their policies and procedures reflect the date collection requirements to ensure consistency in collection.

Loss Amount

APS 115 states that banks “must collect information on the gross loss amount [...] and any recoveries” in their internal loss database. Most AMA banks are able to easily determine the gross loss amount after an event has occurred. However, in instances where the gross loss is not available (i.e. events still under investigation), some banks have recorded a Maximum Potential Loss (MPL) amount. The MPL is a conservative estimate of the potential gross loss amount that is used until the gross loss can be determined. The benefit of including a MPL is that loss events are able to be reported and recorded even if the total loss amount is unknown. Most AMA banks record both gross and net loss amounts (after all recoveries, including insurance). However, currently only losses gross of insurance recoveries are used to quantify the banks ORRC. Banks who wish to use insurance recoveries in their modelling methodology must satisfy the criteria set out in APS 115.

AMA banks have found that losses occasionally materialise that are the result of the same cause event as a loss that is already recorded in the database. Because AMA is essentially an event driven process, it is important that any relationship between individual loss entries is reflected in the loss database and in the risk quantification process. Problems may arise when individual losses fall below the loss collection threshold, but in aggregate they amount to a significant loss. Banks must be aware of such instances and ensure that losses are grouped and reported accordingly.

Currency

In those instances where a bank experiences a loss in a foreign currency (e.g. through an offshore subsidiary), the usual practice has been to record the loss in the database using the domestic currency of the business line in which the loss was experienced. However, for ORRC purposes each loss must be converted into the domestic currency of the group. The most common practice of conversion has been to use the exchange rate at the date of the loss event. However, some banks have used the month-average exchange rate if the timing of the loss is uncertain.

The list above is by no means exhaustive, every aspect of the collection of ILD should be thoroughly documented and independently audited. Policies and procedures should be reviewed and updated on a regular basis to ensure that changes in collection practices are reflected in the relevant policies.

Near Miss/Rapid Recovery and Opportunity Cost Events

As part of the ORMF, ILD is a key component of a bank’s operational risk profile and as such banks must ensure that all material losses are recorded in their operational risk database (APRA, Attachment B p.17, 2008). Although no official definition of materiality exists, banks must demonstrate that their data collection processes are comprehensive and robust. To address this issue, the scope of ILD should be commensurate with its overall use in the ORMF. This includes deciding whether to collect strategic and reputational risk losses and near miss/rapid recovery and/or opportunity cost events. Although the collection of near miss, rapid recovery and opportunity cost events are not mandated by APRA, many AMA accredited banks have incorporated the collection of such information in their internal loss database for risk management purposes and use in scenario analysis workshops.

  • A near miss is “an event, a sequence of events, or an observation of unusual occurrences that possesses the potential of improving a system’s operability by reducing the risk of [loss,] some of which could eventually cause serious damage” (Mürmann and Oktem, 2002).
  • An opportunity cost is defined as “the cost forgone by deciding a particular course of action.” (Bishop, 2004)
  • A rapid recovery is where a loss is incurred but is recovered within a pre-defined period of time (Usually within 48-72 hours). For example a bank incorrectly transfers money into an external bank account which is returned within 24 hours due to an existing relationship with the external bank.

Banks must decide whether the added cost of collecting such information is warranted by its use in the overall risk measurement and management process. Near miss and rapid recovery data tends to be collected more broadly than opportunity costs, predominately because of the difficulty in consistently quantifying opportunity costs.

Given the scarcity of data, many AMA banks have identified that near miss and rapid recovery events contain valuable information on the effectiveness of controls and as such, can be used as an informational input into the scenario analysis process. The challenge in collecting near miss data is that “internal data collection processes typically start with the recognition of a loss event before proceeding to identifying the nature of the operational failure,” (BCBS, October 2006) and because no actual loss is experienced, near miss events are more difficult to consistently identify. Rapid recovery losses are more easily identifiable than near misses because an actual loss is experienced, although recovered a short time later.

Strategic and Reputational Risk Related Losses

Strategic and reputational risk related losses are not widely collected by AMA banks because they are excluded from the Basel definition of operational risk, however they are included as part of Pillar II. Gallati (2003), comments that “reputational risk is one of the key hazards for financial services companies, for which a good name is often a key intellectual property asset. Damage to that good name, is one of the most difficult risks to overcome: you usually can’t pay a fine or take a charge that will quickly reduce the risk to your firm’s reputation, and as such quantifying and capitalising reputational risk is no easy matter.” The British Bankers Association (BBA) has developed a qualitative scale to assess the ‘severity’ of reputational risk losses:

 

​Scale Reputational Impact Definition​
​0 ​No external effect.
​1 ​No media coverage, increase in customer complaints.
​2 ​Limited local or industry media coverage, increase in complaints, possible account, and no negative effect on share price.
​3 ​Limited national media coverage, large scale customer complaints, some customer loss, informal regulatory enquiry, potential negative effect on share price, possible senior management involvement.
​4 ​Sustained national and limited international media coverage, serious customer loss, formal regulatory investigation or enquiry, negative impact on share price, seniow management involvement.
​5 ​Sustained negative national and international media coverage, large scale customer loss, formal regulatory intervention and fines, significant effect on share price, direct senior management/board involvement.

Source: British Bankers Association (BBA).

Using the above scale, banks have a means by which to classify reputational losses without trying to quantify the impact.

Allen (2007) comments that “there is no commonly accepted definition of strategic risk.” APRA has defined strategic risk in their discussion paper ‘Implementation of the Basel II Framework – Supervisory Review Process’ as “external risks to the viability of the ADI from unexpected adverse changes in the business environment with respect to the economy, the political landscape, regulation, technology, social mores and the actions of competitors.” Allen (2007) recommends that banks tailor their definition to focus on the threats to the strategic planning framework and include “external risks to the viability of the business arising from unexpected adverse changes in the business environment with respect to: the economy; political landscape; law and regulation; technology; social mores; and the actions of competitors.”

 

Threshold Determination

The determination of the various thresholds is an important part of the measurement of operational risk due to its significant impact on ORRC (BCBS, October 2006). An internal loss collection threshold is a dollar amount, over which banks collect operational risk losses. This is distinct from a modelling threshold which is the level over which, all losses will be included in the modelling process. Because the collection threshold level is essentially setting the level above which all losses are deemed material “it is important to know the levels of risk in a bank’s operations and their relationship to other operational risks,” (Akkizidis and Bouchereau (pg. 204, 2006)). Banks who set their collection threshold too high may lose valuable information that may have a significant impact on capital. On the other hand banks which set their threshold too low, may find the added cost outweighs the benefit. In determining an appropriate threshold level, most AMA banks have used expert judgement and qualitative techniques rather than more empirical means. APRA has outlined key criteria that must be taken into account during the determination of a bank’s loss data collection threshold, including:

  • The use of ILD in the operational risk measurement system;
  • Availability of data to justify the predictability and stability of Expected Loss (EL) offset amounts;
  • The use of ILD for operational risk management purposes;
  • The amount of administrative requirements placed on business lines and operational risk resources as a result of the data collection and management process.
  • In some banks, business units collect losses below the group collection threshold to increase the data’s usability with respect to risk management and assessment of EL.

 

Expected Loss

ELs are the “highly predictable and reasonably stable,” low severity losses that banks experience in the day to day operations (APRA, 2008). AMA banks are able to obtain a capital deduction for EL provided they are able to demonstrate they have adequately measured and accounted for EL. The UK FSA provides essentially three interpretations of EL (FSA, 2005)):

  1. Business/management – related to a future amount of expense/loss that is predicted on the basis of past experience (Typical Loss).
  2. Mathematical – mean of a loss distribution (Mean Loss).
  3. Financial accounting – losses expected from identified events, for which a reserve has been established. A common example of this is where a large legal cost is anticipated, but the exact amount of the legal settlement is not yet known.

Institutions are likely to use the definition of expected loss which reflects their measurement methodology. Generally speaking, “Typical Loss (TL) is the most pertinent definition for accounting for business practices,” (FSA, 2005) whereas the Mean Loss (ML) should be seen as a cap on EL projections. The difference between TL and ML will vary between institutions and risk types; the diagram below is a graphical representation of EL definitions 1 and 2.

a graphical representation of EL definitions 1 and 2

Because APRA allows flexibility for banks implementing AMA as accepted under the Basel framework, there has been no general consensus on a method for determining operational risk capital and as such, there is no typical method for calculating EL.

Filling in the Blanks

Most banks use ILD explicitly in their operational risk measurement process, and as such the details of each loss become an important component of the overall collection process. APS 115 does not prescribe the depth of information to be collected for each data point, as this will depend on the overall use of ILD in the ORMF. Generally AMA banks require business units to record loss information in an internal database. This method ensures the information used to calculate the business unit’s capital (i.e. loss amount, Basel business line/risk type and date) is recorded and approved by the business unit. However, it does increase the need for independent validation of the data.

The Basel loss allocation framework is made up of a combination of 8 business lines and 7 event types (see Appendix 1). APRA requires AMA banks to report their losses using the Basel framework. Some AMA banks have found it difficult to map losses in circumstances where a loss event may fit naturally into more than one category or has occurred in a centralised group service (i.e. Human Resources). To assist in these situations, Samad-Khan (2002) suggests that a payoff matrix could be used whereby losses are allocated according to:

  • Who benefits (or who was intended to benefit)
  • Who loses (or who was intended to have lost directly or economically).

Samad-Khan (2002) a payoff matrix

For example, losses categorised as ‘Clients, Products and Business Practices’ (CPBP) may be classified by the definition “all types of events committed by a firm employee, where the individual is intending to benefit the firm at the expense of some other party,” there may also be instances where there is no intended sufferer. Using Samad – Khan’s method, CPBP is characterised by the payoff matrix above.

AMA banks have generally used their own methods for allocating losses to the Basel categories, including the use of decision trees. Consistency is the key issue when allocating losses to event types to ensure that the categories are accurately represented in the bank’s risk profile.

Recognition of Insurance

It is possible for AMA accredited banks to include insurance into the calculation of ORRC with written permission from APRA. Banks wishing to include insurance “must be able to demonstrate that the insurance will cover potential operational risk losses included in the operational risk measurement model in a manner equivalent to holding ORRC” (APRA, 2008). Because insurance payouts could be seen to act as substitute capital, banks need to ensure that any insurance recoveries are received in a timely manner to maintain liquidity. Insurance providers have been working with regulators and AMA banks to introduce AMA compliant insurance products which aim to reduce policy ambiguity to ensure speed and certainty of payment. In the past, limited options were available for banks who wanted to insure against low frequency high impact losses (LFHI) (Bunge, 2002). In addition, increasing demand has led to the development of risk mitigation products designed to cover banks in the event of a severe loss (e.g. catastrophe bonds). However providers are still experiencing difficulty in making traditional policies AMA compliant and financially viable (AON 2008).

Data Integrity and Validation

Because the primary drivers for each AMA bank’s capital will depend on their ORMF, it is important for banks to tailor their data validation to their own methodology. Most AMA banks have established model validation policies in place that cover data inputs, model methodology and model outputs. A sound validation process usually encompasses (AQEG, 2006):

  • •Completeness/Coverage of the data
  • Accuracy of the data
  • Relevance of the data
  • Use of the data
Data validation should be an iterative and dynamic process (AQEG, 2006), which changes with respect to industry practice and adjustments to the measurement methodology.

Validation methods vary from bank to bank and include;

  • Manual consistency checks by the risk management function
  • Automated validation during data input
  • Reviews by internal and external audit
  • Reconciliation to the general ledger.
  • Audit tracking of any changes

There is no one best method for data validation, instead banks use a combination of validation tools to ensure the integrity of the data used in their ORMF.

Conclusion

Under Basel II ILD is an essential input of the measurement and management of operational risk, and as such the collection and utilisation of the data are important factors banks (both AMA and standardised) must consider as part of their ORMF.

Polices and procedures are an important aspect for the development of an ORMF and play a key part in disseminating information about the collection of ILD to those who have roles and responsibilities in the risk management process. Policies communicate information about the scope of the data, the type of events that should be recorded and act as a reference point for those unfamiliar with the process. Sound and comprehensive data collection practices mean banks are able to move to more quantitative methods to calculate threshold levels and ensure that their regulatory capital is representative of their internal risk profile. Such data also allows banks to incorporate insurance into their measurement process and to use EL offsets in their ORRC.

The issues mentioned in this paper are by no means exhaustive and simply highlight current industry thinking. AMA accredited banks must continually adapt their methodology to include industry best practice and improve their ORMF and data collection procedures using the experiences learned through the accreditation process and research.

Appendix 1 – Basel Event Allocation Framework

Appendix 1 – Basel Event Allocation Framework

References

Akkizidis, I. and Bouchereau, V. (2006), ‘Guide to Optimal Operational Risk & Basel II,’ CRC Publishing

Allen, B (2007), ‘The Best Laid Plans…,’ Risk Australia, Incisive Media Ltd 2008 (September)

AMA Quantitative Expert Group (AQEG) (2005), ‘Data Integrity and Validation Issues,’ Working Paper (December)

AON (2008), ‘Unlocking the Value of Insurance,’ Operational Risk Symposium (April)

Australian Prudential Regulation Authority (APRA) (2007), ‘Implementation of the Basel II Framework –Supervisory Review Process,’ (September)

Australian Prudential Regulation Authority (APRA) (2008), ‘Prudential Standard APS 115 – Capital Adequacy: Advanced Measurement Approaches to Operational Risk,’ (January)

Basel Committee on Banking Supervision (BCBS) (2005), ‘The treatment of expected losses by banks using the AMA under the Basel II Framework,’ Basel Committee Newsletter No. 7 (November)

Basel Committee on Banking Supervision (BCBS) (2006), ‘Basel II: International Convergence of Capital Measurement and Capital Standards: A Revised Framework – Pillar 1,’ BCBS Publications, Bank for International Settlements (June)

Basel Committee on Banking Supervision (BCBS) (2006), ‘Observed Range of Practice in Key Elements of Advanced Measurement Approaches,’ BCBS Publications, Bank for international Settlements(October)

Bishop, M (2004), ‘Essential Economics,’ Bloomberg Press (April)

Bunge, A (2002), ‘Operational Risk Insurance – Treatment under the New Basel Accord,’ International Finance Seminar (Spring)

Committee of European Banking Supervisors (CEBS) (2004), ‘The Application of the Supervisory Review Process under Pillar 2,’ Consultation Paper No. 3 (May)

Committee of European Banking Supervisors (CEBS) (2006), ‘Guidelines on the implementation, validation and assessment of Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches,’ (April)

Financial Services Authority (FSA) (2005), ‘Treatment of Expected Losses in Capital Calculations – Draft,’ FSA AMA Quantitative Expert Group (May)

Gallati, R. (2003), ‘Risk Management and Capital Adequacy,’ McGraw Hill Publishing

Kumar, T. (2006), ‘The Essential Elements of an Operational Risk Policy,’ i-flex Consulting (March)

Medappa, P. (2006), ‘Operational Loss Events: Data Collection Challenges,’ African Review of Business and Technology (December)

Mürmann, A. and Oktem, U. (2002), ‘The Near-Miss Management of Operational Risk,’ University of Pennsylvania (July)

Samad-Khan, A. (2002), ‘How to Categorise Operational Losses? – Applying Principles as Opposed to Rules,’ OpRisk Analytics, LLC

 

 

1 Authorised Deposit-taking Institutions (ADIs) are corporations authorised under the Banking Act 1959. ADIs include banks, building societies and credit unions.

2 In this paper the term ORMF refers to both the management and measurement of operational risk.