This paper sets out the response of the Australian Prudential Regulation Authority (APRA) to the Basel Committee’s report, Credit Risk Modelling: Current Practices and Applications, issued in April 1999.
In Australia, as in most countries, capital adequacy regulation of banks and other regulated deposit-taking institutions is based on guidelines contained in the Committee’s Capital Accord. APRA therefore takes a close interest in the evolution of the Accord and is keen to contribute to its on-going development. In light of rapid advances in methods for measuring and managing risk in recent years (and the consequent pressures that are building on the existing regulatory framework), APRA welcomes the decision to review the Accord and the Committee’s report on credit risk modelling which forms a part of that process. Broadly, we are in favour of searching for more sophisticated approaches for determining regulatory capital requirements that better reflect differences in risk profiles. Such measures offer the potential for more certain capital protection across different institutions and through time, particularly given banks’ growing capacity not only to detect regulatory anomalies but also to act on that information.
The Committee’s report on credit risk modelling provides a comprehensive listing of the issues currently facing model users (including prudential regulators looking for means of enhancing the regulatory toolset). We agree with the report’s main conclusion that significant hurdles need to be overcome before banks’ internal portfolio credit risk models could be incorporated formally into capital adequacy regulations. Apart from the conceptual and practical difficulties outlined in the report, a shift towards utilising banks’ internal credit models would need to be accompanied by explicit measures for dealing with other risks that currently are only implicitly captured by the existing capital rules. In this regard, banks’ own methods for measuring these risks tend to be less developed and face even greater data availability and validation issues than their credit risk counterparts. Account would also need to be taken of the impact of any regulatory changes on the position of non-model using institutions. We note that consideration of these other issues is already under way and is a focus of the reform proposals outlined in the Committee’s consultative paper on a new capital adequacy framework issued in June.
That said, APRA is of the view that the development and use of portfolio credit risk models within the banking industry, even if not formally incorporated into the regulatory framework at this time, offers important benefits for banks and their supervisors that should not be ignored. Consequently, we believe that supervisors should play a role in encouraging the continued development of these models.
This paper comprises two parts. The first is an outline of the current state of play in relation to the portfolio credit risk models used by Australian banks. While at this stage portfolio credit risk modelling is only performed by the four largest Australian banks, we believe that these banks’ models are at least as well developed as those in most other large international banks. The second part of the paper provides some of APRA’s current views on the development of supervisory guidance to aid the banking industry and help develop consensus on good modelling practice. The main point to be made is the need for further international cooperation and research by supervisors on a wide range of topics in the area of credit risk modelling.
APRA hopes the Committee finds this submission to be a helpful contribution to its deliberations on credit risk modelling, and remains ready to assist in future work on this topic.
Like other large banks around the world (several with operations in Australia), the major Australian banks have been directing substantial resources towards developing portfolio credit risk models in recent years. Briefly characterising the operations of these institutions, the assets of the four major local banks range from around AUD130 billion to AUD250 billion (approximately USD 85 billion to USD160 billion), placing them among the world’s largest 100 banks by asset size and 75 largest by Tier 1 capital. These banks provide (either directly or via subsidiary operations) a wide variety of financial services, from the more traditional banking products to operating in securities, funds management and insurance markets. They account for a large slice (65 per cent) of local banking assets and have a significant offshore presence, with the international networks of three of the banks accounting for between 25 per cent and 50 per cent of their group operations.
In the late 1980s and early 1990s, several Australian banks experienced major asset quality problems. This led to losses at some institutions and severely impacted profitability at others. As a result of this experience, all banks devoted considerable resources to reconfiguring their credit risk management frameworks. This included enhancing the independence of credit decisions, improving credit risk assessment and monitoring, and adopting a more rigorous approach to problem loan management. Over the past few years, the major Australian banks have taken this process further by drawing on concepts initially used for quantifying market risk to build portfolio credit models.
Output from the local banks’ portfolio credit models, generally in combination with other risk models, is used for a variety of purposes, usually including:
- benchmarking total capital and general provisioning levels;
- internal capital allocation (including by business unit, geography, product, distribution channel, customer and individual transaction);
- risk-adjusted profitability measurement (using risk-adjusted return on capital and shareholder value added concepts);
- performance-based remuneration;
- risk-based pricing;
- internal risk reporting; and
- the setting of credit risk concentration limits.
In addition, the local banks are looking to manage more actively their credit portfolios. This includes utilising new opportunities for transferring and trading credit risk presented by credit derivatives, securitisation and secondary loan markets. However, while banks’ credit models can assist portfolio managers to judge directional improvements in risk and return from proposed portfolio fine tuning, model simplifications (particularly in relation to correlations among exposures and the modelling of highly structured exposures) often mean that existing models remain fairly blunt instruments for portfolio management purposes.
The major Australian banks, like their overseas counterparts, have mostly implemented in-house developed default-mode, rather than mark-to-market (also referred to as mark-to-model or MTM), portfolio credit models. With expanding opportunities for trading credit risk, advances in modelling techniques and more widespread familiarity with credit risk modelling concepts, this situation is currently set to reverse in the near term. The shift to MTM models will result in greater reliance on external model vendors, in particular KMV Corporation.
As also noted in the Committee’s report on credit risk modelling, banks often apply different modelling approaches to different types of credit exposures. Usually a more aggregated approach is applied to the Australian banks’ delinquency-managed (retail) exposures. Some banks separately model such exposures on the basis of a top-down, rather than bottom-up, analysis of their credit loss histories. Other banks incorporate such exposures into their main bottom-up models by bucketing the exposures according to loan type, borrower’s industry of occupation and/or geographic location. Default characteristics are then assigned on an aggregated basis. As in all areas of credit risk modelling, these techniques often involve a good deal of judgement and data series of relatively short duration which, in some parts of each group, are proxied by the bank’s experience in other locations. Where certain types of exposures are separately modelled, methods of aggregating the models’ outputs also differ. One approach simply adds the results together; however, this ignores the potential benefits of diversification if the various portfolios involved are not perfectly correlated with each other. Another approach combines the models’ results based on an assumed degree of correlation among the relevant portfolios; this is typically a judgemental rather than empirically-based exercise.
Consistent with overseas experience, the Australian banks have generally adopted a one-year credit risk modelling horizon. One bank deviates from its peers in that it uses multiple modelling horizons based on the different tenors of its credit exposures. Similarly, target insolvency rates (or, equivalently, risk coverage levels) are chosen consistent with each bank’s desired credit rating. Although all of the major local banks currently target a ‘AA’ rating, small differences in the way each bank quantifies its desired risk coverage level can contribute to large differences in credit risk capital calculations – in the order of several hundred million dollars for each basis point step in probability.1
Currently, none of the Australian banks favours a credit risk modelling approach conditioned on the state of the economy. Apart from the additional model complexity involved, the banks express concern that errors in forecasting economic turning points could lead, in particular, to a shortfall in desired capital coverage just as the economy turns sharply downwards.
A central feature of the Australian banks’ portfolio credit models is their use of internal credit risk grading systems to assign default (and other credit risk migration) probabilities to their transaction-managed credit exposures. Combined with the increasing incidence of risk-adjusted employee remuneration schemes, this has given added impetus to the banks’ efforts to strengthen their credit risk rating processes over recent years. Among other things, each of the major banks has either converted, or is in the process of converting, its original credit risk rating system into a two-dimensional or ‘composite’ system. Composite systems separately assess both the likelihood of a borrower defaulting and the likely severity of loss should default occur. Compared with the banks’ original systems which intermingled these concepts in a single rating, composite rating systems are better aligned conceptually with the banks’ portfolio credit models, and provide more useful information for general credit management purposes. Most banks are also considering introducing greater granularity into their rating scales by increasing the number of grades. In some cases, this will involve ratings scales with close to 20 separate credit grades.
The local banks have also introduced a range of measures designed to enhance the accuracy, integrity and consistency of ratings throughout their operations. Generally speaking, these measures seek to reduce the degree of subjectivity and/or inject greater independence into their ratings processes. For example, the major banks have introduced a variety of rating assessment tools to assist staff in their rating determinations, including use of external ratings (where these are available) and debt rating models. Existing debt rating models range from expert knowledge-based to statistically-based models; the latter either developed in-house for various segments of the banks’ credit portfolios or purchased from external vendors. Most of the statistical models are actuarially based, ie the models utilise historical data on defaulted and non-defaulted borrowers to rate customers with similar characteristics, though equity-based models, which infer default probabilities from borrowers’ liability structures and stock-price information, are also used. As also noted in the Committee’s report, most banks tend to use these latter models as a supplementary means for cross-checking the results of their main ratings tools in rating appropriate segments of their portfolios. Some banks are also exploring the usefulness of other statistical techniques, such as neural networks for this purpose. At the same time, several banks are re-examining the role of subjective inputs in their models. While there is evidence that such data are predictive, subjective inputs are also more open to interpretation/manipulation and can be less responsive to changed borrower circumstances.
Credit approval/review policies are another means by which the banks seek to enhance the accuracy, integrity and consistency of their rating processes. These policies channel larger and other potentially riskier credit exposures into the bank’s independent credit line and/or to higher approval levels within the credit line, require a higher level of approval where lending/credit officers propose overriding the rating recommendations of the bank’s rating models, and seek to enhance the timeliness of rating adjustments by specifying early review events and/or more frequent periodic reviews for lower-rated exposures. Banks have also strengthened the periodic review of assigned credit ratings by independent credit review teams (which also report main trends to top management levels within the bank and to board risk management and internal audit committees) and are increasingly using poor credit process ratings received from independent credit review teams to influence employee compensation. There is also regular monitoring of credit migration data against expected outcomes.
In order to reduce inconsistencies among their various data management systems, some banks have established, or are considering establishing, automated data transfer linkages designed to minimise/eliminate rekeying of ratings input data and of completed ratings. Some systems also seek to track potential ‘gaming’ of rating models, eg where loans officers enter customer information several times in order to obtain a better credit rating. Only one bank, however, regularly undertakes centralised monitoring of model override trends, even though such monitoring can help indicate potential problems in the way rating models are being used within a bank and/or deterioration in model performance.
In line with overseas experience, the local banks currently lack long-term data on the performance of their credit ratings systems. In order to assign quantitative default probabilities to their credit exposure, the banks have either aligned their risk grades with the 12-month rates predicted by their debt rating models or with the experience of longer-established external rating schemes (ie Moody’s and/or Standard Poor’s). Differences in the choice of benchmark have led to a good deal of variation among the banks’ probability scales. Benchmarks based on external experience also give rise to questions about their applicability to a bank’s particular client base and lending practices. Ultimately, judgements about the appropriateness of any particular benchmark become an empirical question. At this stage, the banks’ rating systems appear capable of ranking risk in an ordinal (or relative) sense. However, the accuracy of their associated cardinal probability scales remains problematic as available credit migration data have only been drawn from the good part of the current credit cycle.
A second major input into banks’ portfolio credit models are loss-given-default (LGD) rates. As in other areas of credit risk modelling, the banks’ treatment of LGD rates is heavily constrained by data limitations. Although the major Australian banks are working to reduce some of these limitations, key LGD parameters are currently a mixture of the banks’ own loss histories (mostly restricted to data on Australian-based customers and covering a limited time span), published studies of bond and commercial loan experience (mostly data on US-based customers) and management judgement. When ascribing LGD characteristics to credit exposures, the banks usually group their exposures on the basis of security coverage and, in some cases, also by product type or business line. One bank buckets its exposures according to size.
After controlling for security coverage etc, some overseas banks collapse LGD distributions into single numbers (usually cyclical averages). As LGD distributions tend to be widely dispersed, this practice can substantially reduce the modelled variability of a bank’s overall credit loss distribution (and its economic capital requirement). In contrast, the Australian banks generally seek to capture LGD volatility in their economic equity calculations. In modelling LGD rates as stochastic rather than deterministic variables, the banks generally assume that LGD distributions are well described by beta distributions (also a common practice overseas). One bank utilises empirical distributions derived from its own loss history.
A third important element in banks’ credit models is the way in which interrelationships among credit exposures are handled. The treatment of correlations among credit exposures determines how well risk concentrations within credit portfolios are identified within the capital attribution process. Correlations are difficult to estimate accurately. Banks that use bottom-up modelling techniques (including the Australian banks) typically assume zero correlation among most of the risk factors recognised in their models. Although the impact of default correlation (and, in MTM models, correlation among other credit rating changes) is typically estimated in order to introduce portfolio diversification effects more realistically into the models, the zeroing out of other correlation effects ignores the tendency for default probabilities, LGD rates and credit spreads all to increase together during economic downturns. Potentially, this could result in substantial underestimation of the likelihood and magnitude of extreme events and, consequently, overestimation of the level of protection provided by particular amounts of capital.
The Australian banks incorporate default correlation into their models in a variety of ways. Some banks assume, either explicitly or implicitly, average correlations across the whole or large parts of their credit portfolios. While this approach is sensitive to large single obligor exposures, it is insensitive to the build-up of industry and geographic risk concentrations. Other banks estimate credit quality correlations based on a multi-factor analysis of world equity market prices developed by KMV Corporation. While the latter approach takes into account industry and country influences, the impact of high-correlation events (such as the ‘Asian crisis’) can be swamped by averaging effects over the model’s longer-run observation period, various industries/countries may be under-represented in the stock price indices used in the estimation process and/or correlations among smaller borrowers may be different to those among listed firms. The potential importance of these issues will, of course, depend on the structure of particular banks’ portfolios.
For banks heavily involved in speculative asset financing, for example, the above issues are likely to be of particular concern. The asset price bubbles experienced by many economies in the late 1980s and early 1990s provide a good example. The subsequent sharp downturn in asset prices compounded overall losses by not only contributing to a higher level of defaults but also to lower recoveries from the sale of collateral.
Other modelling variations arise out of the ways in which the banks incorporate various types of exposures into their credit models. For example, one bank applies 12-month default probabilities to all of its credit exposures. The other banks use different probabilities depending on the tenors of their exposures; this results in higher equity being attributed to longer-dated (generally more risky) exposures. In addition, a variety of methods are used to deal with credit exposures whose magnitudes can vary over the modelling period for credit-related and other reasons, including:
- all of the major banks assume that exposures with maturities shorter than their model’s time horizon are rolled over;
- some banks assume that currently undrawn commitments will be 100 per cent utilised in the event of default while other banks assume that only a proportion of such limits will be drawn down (the factors used by the latter banks tend to be based on published, rather than internal, studies of limit usage in default and on management judgement);
- one bank uses the credit conversion factors contained in the regulatory capital framework for contingent exposures such as performance guarantees; the other banks use 100 per cent of the relevant facility limits (though some of the latter banks are reviewing their approach);
- all of the banks have different methods for converting derivative exposures into credit equivalents. As with contingent exposures, one bank uses the credit conversion factors in the capital adequacy framework, while others use more sophisticated approaches to estimate potential exposure. No bank is currently integrating its credit and market risk models to estimate jointly the credit and market risk exposures for these instruments;
- only one bank attributes capital against its foreign exchange settlement risk. Only one local bank attempts to model cross-border risk. We understand that a similar situation exists overseas though interest in this area has picked up following the onset of problems in Asia.
Aggregation methods also differ among the banks. Because of the complexity of credit models, simulation techniques tend to be used or a mean-variance approach is taken where the shape of the credit loss distribution is simply assumed. Banks often use both methods, eg simulation techniques are applied to their transaction-managed exposures while a top-down mean-variance approach is applied to their delinquency-managed retail exposures. Where mean-variance approaches are utilised, the banks have generally settled on different distributional shapes to represent the overall portfolio loss distribution (eg lognormal, gamma, inverse normal distributions). While each of these distributions results in the attribution of different amounts of capital, there is usually little empirical or theoretical basis for the banks’ choices.
More investigation needs to be undertaken into the materiality of differences in the banks’ modelling approaches. After adjusting roughly for portfolio size differences, the credit risk capital calculations of the major Australian banks differ by around 25-30 per cent from one another. These differences reflect both differences in measurement methodologies as well as real differences in risk profiles; almost certainly, the numbers also mask offsetting differences in measurement methodologies among the banks.
On the issues of stress testing and sensitivity analysis, the Committee’s description of overseas practice applies also in Australia. Generally, the Australian banks have not investigated model sensitivity to key parameter changes or sought to quantify overall potential estimation error. Also, to date, stress testing has tended to be sporadic and relatively unsophisticated.
The development of sophisticated credit modelling approaches in Australian banks appears to be at a similar stage to that of their international counterparts. However, as is also the case overseas, these approaches are evolving and subject to severe data limitations, which not only require the use of important simplifying assumptions and proxy data but also impact on the banks’ ability to validate the overall accuracy and robustness of their models.
Irrespective of the ultimate prospects of utilising banks’ portfolio credit risk models for regulatory purposes, from a supervisory perspective the models can play a potentially important role in:
- strengthening banks’ credit risk management capabilities;
- providing additional insights that can be incorporated into supervisory assessments of institutions’ capital adequacy and risk management capabilities; and
- contributing, through enhanced disclosure, to the market’s understanding of risk management practices and profiles amongst the more complex and sophisticated banks.
For these reasons supervisors have a role in encouraging the continued development of credit (and other) risk modelling techniques. As many aspects of model development are still in their infancy, supervisors need to be extremely cautious about becoming too prescriptive too soon. However, a common point raised by banks with whom we have discussed modelling issues has been a desire for supervisors to play a role in this development by providing guidance to the industry on good practice and minimum standards.
The Committee acknowledges that, for portfolio credit models to be used for regulatory purposes, qualitative and quantitative standards might need to be developed by supervisors. Based on its discussions with Australian banks, APRA is of the view that such standards undoubtedly will be required. As banks continue to develop their modelling capabilities, it will be important that supervisors provide guidance wherever possible to ensure that the evolution of these models remains mindful of possible future regulatory needs.
With the above in mind, some preliminary views on a number of issues are outlined below.
- Most of the major Australian banks currently operate default-mode models though some banks have indicated they are likely to move to an MTM approach. Banks usually justify their use of default-mode models by noting that the bulk of their credit exposures are illiquid and held to maturity. They argue that notional changes in loan values resulting from credit quality changes short of default are therefore irrelevant. In its credit risk modelling report, the Committee also suggests that model suitability can be assessed largely by reference to the purposes to which the model is put. For example, a bank that utilises credit risk models for performance measurement purposes associated with a buy-and-hold portfolio might reasonably opt for a (simpler) default-mode model while a portfolio comprising more liquid credits might require a more comprehensive loss measure incorporating potential shifts in credit quality and spreads.
We tend to the view that, for the purposes of assessing capital adequacy from a regulatory perspective, the MTM paradigm is more relevant. While the default-mode approach may be appropriate for judging performance on a ‘going-concern’ basis, the MTM approach seems better suited conceptually to assessing the capital coverage provided in potential stress situations when an institution may be looking to liquidate its book. This suggests that some sort of additional capital buffer would need to be applied to those banks using default-mode models if the models were to be used for regulatory capital purposes at some stage in the future. In the meantime, research into the typical order of magnitude of differences in the output from default-mode and MTM models could provide useful interpretive information for supervisors undertaking their prudential assessments and when comparing the outputs from different models.
- Worldwide, most banks apply a one-year credit risk modelling horizon. The usual choice of a 12-month time horizon tends to reflect data availability issues and fits in with banks’ planning/budgeting time frames. The banks also argue that a year represents a reasonable period over which risk-mitigating actions could normally be taken to reshape their business activities and/or recapitalise their operations.
Existing risk models assume that banks can be recapitalised at any time with their ability to raise additional capital unaffected by prior-period events. Experience suggests, however, that normally available courses of action (including repeated capital raisings) may be closed to institutions under severe stress, with problems at troubled banks often taking several years to resolve. In addition, annual fluctuations in economic activity and credit losses tend to be serially correlated. Together, these factors suggest that banks’ capital buffers might be required to absorb significant credit shocks for periods extending well beyond one year. In our view, this points to the need for longer modelling time horizons than the current industry standard of one year for most banks’ portfolios, perhaps related more to the typical length of an economic downturn or workout situation.
As noted in the previous section, some banks use multiple time horizons in measuring their credit risk capital requirements. This can be justified on the basis of factors such as business strategy or differing liquidity characteristics of particular products or markets. Where such an approach is taken, careful consideration needs to be given to whether (and if so, the extent to which) any diversification benefits between the relevant portfolios should be incorporated into the model.
- Both published and banks’ internal studies indicate that LGD rates are highly variable and subject to strong cyclicality. At a minimum, banks should be seeking to take LGD volatility into account in their credit risk models. Banks which treat LGD rates as non-stochastic run the risk of substantially underestimating the variability of their overall credit risk distributions and consequently overstating the risk coverage levels provided by particular amounts of capital.
The issue of cyclicality is important but more difficult given the lack of good long-run data. While it seems unlikely that banks will be able to relax the usual assumptions of independence among LGD rates, and between LGD and default rates, in the short term, investigation of the impact of these assumptions could provide useful interpretive information for supervisors and bank model users.
- The difficulties involved in validating portfolio credit risk models are well documented. While we have little to add to the observations contained in the Committee’s report on this issue, we note that banks’ overall economic equity calculations (ie covering all risk types not just credit risk) tend to fall well below the regulatory benchmark of 8 per cent of risk weighted assets.
Over the past decade, experience in a number of countries has shown that even the current requirements do not necessarily prevent banks from experiencing substantial difficulties in the event of major economic shocks. In the period of high credit losses experienced in Australia at the start of the decade, for example, several banks saw their capital bases eroded substantially in a very short period of time. Since that time, credit risk management practices have changed. Among other things, portfolio concentrations tend generally to be less pronounced. The Committee could consider co-ordinating an investigation into the capital levels that banks’ credit risk models would have required in the lead up to the banking problems experienced in a number of countries since the late 1980s. Although it may prove difficult to reconstruct balance sheets in sufficient detail, such a study could potentially provide useful information about the robustness of banks’ credit risk models. For example, to what extent would the modelled capital numbers for earlier periods have closed the gap that currently exists between banks’ economic and regulatory capital levels?
- We note the Committee’s finding that the majority of banks surveyed for its risk modelling report assign ratings to counterparties solely according to the judgement of a loans officer. We find this surprising given the importance of risk grading systems in the modelling process. It is difficult to see how a purely subjective rating system could be acceptable for regulatory purposes, particularly if used for a large part of a bank’s credit portfolio.
– As is the case elsewhere in the world, all Australian banks are relying, at least in part, on third-party providers of data, software, and analysis tools in their construction of credit risk models. In their own reviews of banks’ modelling activities, supervisors therefore also need to review the applicability and credibility of model components supplied by these providers. It seems sensible to develop mechanisms by which supervisory bodies can exchange views on the merits and shortcomings of these products. A good example is widespread use of KMV Corporation’s models: a more co-ordinated approach to the examination of these products would be helpful to both supervisors and the banking industry.
More generally, more thought, analysis and debate will undoubtedly occur on a wide range of issues before supervisors are able to proffer firm views on good practice. Many issues can only be resolved by further empirical work. In this regard, the Committee would appear to have a leadership, co-ordination and information-sharing role in helping to build consensus in this area. As part of this process, the Committee should consider measures that would help foster a broader supervisory network for exchanging information and views at a working level. APRA is prepared to contribute to any initiatives that are established in this area.
APRA received submissions on the Basel Committee’s paper, Credit Risk Modelling: Current Practices and Applications, from the following bodies:
- Australia and New Zealand Banking Group
- Commonwealth Bank of Australia
- National Australia Bank
- Westpac Banking Corporation
- Institute of Actuaries of Australia
While the views contained in this submission are our own, their formulation has benefited greatly from comments received from, and discussions with, the above.
1 Benchmarks for desired coverage include selecting one year default rates from Moody’s or Standard & Poor’s, averaging the data of the two ratings agencies, or selecting default rates for a different time horizon and then scaling this to a one year measure. These methods produce risk coverage levels for a ‘AA’ rating of between 99.93 per cent and 99.97 per cent.