Transforming risk culture: observations from APRA’s pilot survey

Key Points:
 

• APRA’s risk culture survey is internationally leading regulatory practice that expands its supervisory toolkit to transform risk culture. 

• Survey results provide a unique employee view of the risk management practices and behaviours in an entity.

• APRA will benchmark up to 60 more entities in the next 12 months, and will share additional insights with industry.

A strong risk culture is essential for effective risk management outcomes that support an organisation’s financial and operational resilience. Ultimately, organisations with a strong risk culture that supports sound risk management practices and behaviours are better placed in terms of financial performance and fair, quality outcomes for their customers. Under Prudential Standard CPS 220 Risk Management, the boards of APRA-regulated entities are required to form a view of the risk culture in the institution that they govern, and identify any desirable changes to the risk culture necessary to ensure that culture supports the ability of the institution to operate consistently within its risk appetite.

APRA aims to reinforce, support and assess the work regulated entities are doing to build and maintain an effective risk culture. To this end, APRA has introduced an industry-wide risk culture survey recently piloted with 10 general insurance entities. The survey is a key initiative that supports APRA’s expanded supervisory toolkit designed to transform governance, risk culture, remuneration and accountability (GCRA) practices across regulated entities.

APRA’s pilot risk culture survey, conducted between March and April 2021, provides insights from employees on perceived risk behaviours and the effectiveness of the risk management structures within their entities. The responses, over time, will determine the extent to which positive changes to risk culture are (or are not) taking place within individual entities, and correspondingly, will identify areas where an entity’s risk culture can be improved.

The survey also provides the opportunity to benchmark results across a number of regulated entities within an industry sector (for example, insurance), providing an opportunity for entity leaders and APRA supervisors to understand how the entity’s results compare to others in its peer group. APRA is one of the only regulatory bodies worldwide that directly collects survey data at an industry level, so APRA’s risk culture survey represents internationally leading regulatory practice.

What is risk culture?

Risk culture refers to an entity’s attitudes and behaviours towards risk management. Specifically, it is the behavioural norms and practices of individuals and groups that shape an entity’s ability to identify, understand, openly discuss, escalate and act on its current and emerging risks. 

A strong risk culture creates an environment where employees are comfortable speaking up and voicing concerns with their leaders. It produces better decisions by ensuring a broader range of views are considered, and allows ideas that present heightened risks to be appropriately challenged during decision-making. It incentivises boards and senior executives to prioritise effective risk management. In doing these things, a strong risk culture helps to deliver better business and customer outcomes for organisations. APRA is committed to enhancing and reinforcing a strong risk culture across all regulated entities.

In particular, an entity’s risk culture is influenced and shaped by two key aspects:

1. Risk behaviours: the observable actions and behaviours of individuals and groups (for example, role modelling, operating practices and symbols, such as discussion of risk management as a standing agenda item in team meetings), and
2. Risk architecture: the formal structures and arrangements that support the management of risks (for example, systems, policies, procedures and governance structures).

APRA’s Risk Culture 10 Dimensions

APRA has developed a framework called the Risk Culture 10 Dimensions to assess the risk culture of regulated entities. The Risk Culture 10 Dimensions articulate the key aspects of an entity’s risk behaviours and risk architecture that contribute to its risk culture. Each of the survey questions in the pilot (approximately 40) aligned with one of APRA’s Risk Culture 10 Dimensions. 

The Risk Culture 10 Dimensions – coupled with the survey results – allow APRA to access comparable data in a consistent way across regulated entities in order to assess and benchmark risk culture.

Figure 1: APRA’s Risk Culture 10 Dimensions

An accessible version of this infographic is available here.

APRA’s Risk Culture 10 Dimensions is not a prescriptive framework, and APRA does not expect entities to adopt it. While the 10 Dimensions framework provides insights into how APRA assesses risk culture, an entity should have a risk culture framework that fits its own particular circumstances (such as its size and complexity). This framework should allow an entity to measure, monitor and report on its risk culture in a consistent and meaningful way.

Survey process

Participant Profile

APRA sent risk culture surveys to every employee at each participating entity, amounting to approximately 11,600 potential respondents in total. Participation in the survey was voluntary with all survey responses anonymised. The average response rate was 62 per cent.

The survey results provide APRA with a unique employee view (compared to a ‘top down’ management view) of a regulated entity’s risk culture. Employees offer an important perspective on an entity’s risk culture, and on management’s initiatives to improve risk-related practices and behaviours. A majority of the survey respondents – 64 per cent – were employees with no management responsibilities, and 46 per cent had been employed with their entity for over five years. 

Figure 2: Pilot Cohort Participant Profile – Management Level

Figure 3: Pilot Cohort Participant Profile – Tenure

Data Quality

APRA included an attention check question in the survey to help assess data quality. The purpose of the attention check is to differentiate between people who provide thoughtful responses – based on paying attention to the content of the survey – and employees who are not paying close enough attention, thereby making the data unreliable and not representative of an entity. Where respondents failed the attention check question, their responses were excluded.

The average attention check failure rate in the pilot survey was 20 per cent, which is noticeably higher than in previous APRA risk culture surveys (i.e. conducted with individual entities in risk culture deep dive reviews). As APRA rolls out the risk culture survey to other entities, it will closely monitor the attention check failure rate as a measure of data quality.

Cohort Benchmarking

In order to benchmark the entities surveyed, a quartile approach was applied. Results for each risk culture dimension were presented as a percentage agreeable: that is, as a proportion of respondents that chose ‘agree’ or ‘strongly agree’ to the statements relating to a particular risk culture dimension. Benchmarking was then determined by assessing the results relative to quartiles; that is top quartile, interquartile range (the two middle quartiles) and bottom quartile results within the cohort.  

Two entities had results that predominately fell into the top quartile, while bottom quartile results were more varied, although generally concentrated across four entities. 

Figure 4: Average % Agreeable Results vs Entity Response Rate

However, a top quartile result does not mean that further improvement isn’t needed. APRA notes that there is scope for top quartile results to improve across all 10 risk culture dimensions. 

Key risk culture survey results

Some of the key observations from the pilot risk culture survey are set out below.

Risk Culture Dimension Analysis

When assessing each risk culture dimension, the lowest scoring dimensions were:
•    Risk Governance and Controls;
•    Decision-making and Challenge; and
•    Responsibility and Accountability. 

These results help surveyed entities identify priority areas that may warrant additional focus within their organisations. 

The lowest top quartile threshold was in the risk culture dimension of Risk Governance and Control. This dimension assesses the effectiveness of risk management oversight through systems such as governance structures, reporting and control processes. This is an area that is critical to building a strong risk culture and effective risk management within an entity.

The Responsibility and Accountability dimension also had the largest variability in responses. This indicates that employees tend to have a wide range of experiences when responding to questions regarding, for example, being clear about the risks they are responsible for managing within their roles. 

Business Area Analysis

As well as comparing between entities, APRA was also able to assess how each business area within the surveyed entity responded relative to other business areas in their organisation, as well as the same business area across all other surveyed entities. The analysis found that the business areas of Underwriting and Customer Service were among the most negative, particularly in relation to the risk culture dimensions of Responsibility and Accountability, and Risk Governance and Controls. These results help entities to target resources to better understand the drivers of diverging employee perceptions and mindsets in these business areas.

Employees in the Financial Control business area, together with those employees in Legal, Compliance and Risk were most positive across all 10 risk culture dimensions. In particular, these business areas had the highest percentage of positive scores in the risk culture dimensions that Underwriting and Customer Service business areas rated lowest. While these business areas are likely to be more positive (by virtue of their risk management-related functions), there is an opportunity for entities to understand what is working well in these areas (for example, training, leadership communication) to potentially apply these to areas where there is less favourable sentiment. 

Using the risk culture survey results

APRA sent each participating entity a detailed report outlining their risk culture survey results. While the survey results provide an important perspective on an entity’s risk culture, it represents only one of a number of qualitative and quantitative approaches used to assess risk culture.

APRA has found the pilot risk culture survey to be a rich source of insights. APRA supervisors will consider an entity’s survey results, together with other supervisory information, in order to strengthen their assessment of a regulated entity’s risk culture.

APRA expects that entities will use the insights from the survey and compare them with their own internal indicators (such as employee survey results, risk culture metrics and internal risk culture reviews), and other data points (such as non-financial risk metrics) to build a more comprehensive picture of their risk culture.

In addition, an entity may want to focus on the specific areas highlighted in their report to gain a deeper understanding of the underlying causes of employee perceptions and mindsets (for example, through interviews and focus groups), so that meaningful initiatives to strengthen the entity’s risk culture can be implemented.
 

Where to from here?
 

APRA plans to roll out the risk culture survey to up to 60 banking, insurance and superannuation entities over the next 12 months, according to the following timeline:

Figure 5: Timeline for survey roll out
 

APRA is refining the risk culture survey questions, together with the analysis and reporting, to ensure that greater reliability, accuracy and interpretation of an entity’s risk culture can be provided.

The risk culture survey supports APRA’s continued commitment to transforming GCRA, while enhancing regulated entities’ risk culture awareness and analysis, as well as strengthening APRA’s own ability to efficiently and effectively assess and benchmark entities’ risk culture.