Skip Ribbon Commands
Skip to main content
 
ARCHIVED CONTENT

APRA Stakeholder Survey 2011

Report of overall findings
July 2011

Executive summary
 
In March and April 2011, the Australian Prudential Regulation Authority (APRA) conducted a study of its stakeholders as part of its service charter commitments. APRA engaged Australian Survey Research (ASR) to deploy and analyse a web survey of 563 regulated entities and 150 knowledgeable observers.
 
The 2011 survey was closely based on a questionnaire developed by both APRA and ASR in the first round of surveying conducted in 2009. The Australian Bureau of Statistics Statistical Clearing House approved the 2011 survey. A total of 65.5% of regulated entities responded which is a statistically representative sample and which closely reflects the population profile on a number of attributes. Sixty-one knowledgeable observers completed their survey which was a shorter version of the regulated entity survey.
 
Within the questionnaire there were 45 rated items which used a five point rating scale. Only two items scored below 3.0 (neutral) on the five-point scale and 23 of the 45 items had 100% or more positive responses.
 
Overall this is an excellent result and further endorsement of APRA's prudential framework and approach to supervision. Regulated entities agree that APRA has had a positive impact on their industry.
 
The highest and lowest scoring items are displayed in the tables below. The blue items were asked only of group entities (n = 114).
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree; while * items used a 5 point never-always scale
 
HIGHEST SCORING ITEMS MEAN
A single supervisory team responsible for all group companies is an appropriate way to supervise groups 4.6
APRA staff’s demonstration of integrity* 4.5
APRA staff’s demonstration of professionalism* 4.4
APRA's enforcement of its prudential requirements has had an impact on your industry 4.2
APRA's framework is effective in achieving its mission 4.2
APRA's guidance material is of value to your organisation 4.2
APRA's harmonisation of the prudential framework is important 4.2
APRA staff’s demonstration of collaboration* 4.1
APRA is effective in communicating the findings of supervisory visits to your organisation 4.1
The APRA supervisory team responsible for your organisation has a good understanding of your organisation 4.1
 
LOWEST SCORING ITEMS MEAN
APRA's PAIRS rating reflects your organisation's view of its risk profile 3.6
Standards & guidance material clearly communicate requirements 3.6
APRA meets its stated approach of being consultative in its supervision 3.6
APRA meets its stated approach of being consistent in its supervision 3.6
The instructions to APRA's statistical forms are helpful 3.6
D2A is easy to use when lodging data with APRA 3.5
The effort required of your organisation during APRA's prudential reviews is appropriate 3.5
During supervisory visits to your organisation, APRA supervisors focus on principles rather than detailed prescription 3.5
Changes to APRA's prudential framework consider the costs of regulation 2.8
APRA has successfully harmonised its prudential framework 2.8
 
There have been very few changes since 2009: only five items were statistically significantly different between the 2009 and 2011 surveys. All other rated-item results between the two years were the same, at a 95% confidence level and within the ±5% confidence interval. This indicates a very stable pattern of behaviour from APRA for those items measured in this survey.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
STATISTICALLY SIGNFICANTLY DIFFERENT ITEMS 2009
n=392
2011
n=369
APRA has successfully harmonised its prudential framework across the industries it regulates 3.4 2.8
During supervisory visits to your organisation, APRA supervisors focus on principles rather than detailed prescription 3.6 3.5
During supervisory visits to your organisation, APRA supervisors focus on major risks or controls 4.0 3.9
A single supervisory team responsible for all group companies is an appropriate way to supervise groups 4.3 4.6
APRA is helpful when your organisation has difficulties using D2A 3.9 3.8
 
In addition to these changes, in 2011 more than three-quarters of entities indicated that the amount of statistical data collected by APRA was about right (79.7%), while 19% indicated that it was too much or far too much. This is a significant improvement on the 2009 survey results when 66.3% of entities indicated that the amount of statistical data collected was about right and 32.1% indicated that it was too much or far too much.
 
APRA's strengths are the impact of its framework and guidance material as well as its staff's adherence to its values. Areas that scored lower include the cost impact of regulation, particularly for small entities, and harmonisation across regulators and across standards.
 
The items that have the greatest impact on an entity's view of how well APRA enforces its prudential requirements are, in decreasing order of impact:
  • APRA's risk assessment of your organisation, as conveyed to you in review reports, is aligned with your organisation's own risk assessment
  • APRA meets its stated approach of supervising in line with international best practice
  • APRA's supervision practices have had a positive impact on your organisation's risk management practices over the past three years; and
  • APRA's guidance material is of value to your organisation.
Where a regulated entity and APRA have common views about the entity's risk profile, APRA is more likely to be viewed positively. The reverse also applies.
 
Across the survey, views differ considerably between industry groups but with no overall pattern to differences, while views vary little by size and whether an entity is related to a group or not.
 
There were considerable differences in scores between knowledgeable observers and regulated entities. A majority of the comparable items are statistically significantly different. Knowledgeable observers tend to rate APRA higher than regulated entities. The exception is around APRA staff's demonstration of APRA's values where knowledgeable observers rated APRA staff significantly lower than regulated entities on all values except integrity, but still at a fairly high level.
 
 
Introduction
 
As part of its published service charter, the Australian Prudential Regulation Authority (APRA) is committed to surveying regulated institutions, industry bodies and other stakeholders to understand the impact of APRA's prudential framework and the effectiveness of its supervision. Within this charter, APRA conducted its first stakeholder survey in 2009.
 
In late 2010, APRA engaged Australian Survey Research (ASR) to conduct a repeat of the 2009 study using an identical method and very similar content. Again two surveys were conducted: one of regulated entities and a shorter one of industry representatives and other knowledgeable observers. This report outlines the methodology used in the surveys as well as the key findings. Detailed questions appear in an attachment to the report.
 
Key findings are presented from an overall perspective, as well as by industry and size and common themes are presented for each free text question. Comparative results between the two rounds of surveying (2009 compared with 2011) are presented and discussed. Findings from industry representatives and knowledgeable observers form a section within this report.
 
 
Methodology
 
This section outlines how the questionnaire was developed and tested, how survey participants were identified and how the survey was administered and analysed.
 
Questionnaire
 
For the original 2009 stakeholder study, APRA project staff prepared a preliminary set of questions that were drawn from other, similar international studies and from key aspects of APRA's Service Charter including its stated corporate values. Together, ASR and APRA further refined the questionnaire and input was sought from a range of internal stakeholders including the APRA Members. The 2011 questionnaire was updated to reflect changes in APRA operations and publications, but only minor additional changes were made. No questions were added or dropped for 2011.
 
Both web questionnaires were loaded into ASR's proprietary web surveying tool, Survey Manager, and hosted on ASR's internet servers located in a high security data centre in Melbourne's CBD.
 
The questionnaire was pilot tested in 2009. For the 2011 study, no pilot test was conducted because there were only minimal changes to the survey conduct and content.
 
The Australian Bureau of Statistics Statistical Clearing House approved the regulated entities survey in concept and execution, including the questionnaire, in February 2011.
 
Data collection
 
APRA provided ASR with a full listing of all regulated entities (n=563), each entity's designated APRA prudential contact, along with the contact's email address and details such as entity name, industry subgroup, size in terms of asset base and attachment (or not) to a regulated group. The survey was a census of APRA's regulated entities.
 
Prior to going live with the full survey, APRA chairman, Dr John Laker, sent a letter to the CEOs of all regulated entities in Australia advising them of the survey. Soon after, ASR sent an email invitation to each prudential contact. The email invitation contained a unique hyperlink to access each entity's questionnaire. The email also contained instructions for the prudential contact to liaise with the APRA statistical contact within their organisation to help complete the questionnaire. In some organisations the prudential contact and the statistical contact were the same person. For the purpose of this survey, each regulated entity was considered a unit within the population. Within the population of 563 regulated entities, 515 entities had a unique prudential contact while 88 prudential contacts were required to answer about three or more entities. One person was a prudential contact for 12 entities.
 
ASR tracked the response rate and sent targeted reminder emails to non-responders.
 
Before an entity's response was finally submitted (completed), the CEO of each organisation was asked to complete a declaration endorsing the answers provided.
 
Data analysis
 
Results have been analysed to produce mean scores (averages) and frequency distributions. Various statistical tests including chi square, t test and ANOVA have been used to determine any statistical differences between demographic sub-groups (such as industry and size). All tests are reported at the p<0.05 level (95% confidence level).
 
Means have been calculated using only the number of respondents who chose a rating point answer. In other words don't know, not applicable and no answers (blank) have been excluded from statistical calculations.
 
The rating scale used to assess most items is displayed in the table below. It is important to understand what the numbers represent because results later in the report are presented in numeric form only. For example, a mean (average) score of 4.0 indicates that, overall, respondents agree that APRA is performing well on a particular item. Some items were asked using a different rating scale. Where a survey item was presented with a different scale to the one shown below, it is noted and explained in the report.
 
RATING SCALE DESCRIPTION ASSIGNED NUMERIC VALUE
Strongly disagree 1
Disagree 2
Neutral 3
Agree 4
Strongly agree 5
 
Sample profile
 
In total 369 stakeholders responded to the survey yielding a response rate of 65.5%. The sample is statistically representative of the stakeholder population as a whole at the 95% confidence level and the ±3% confidence interval (see note below explaining confidence interval and level). This is higher (more rigorous) than a commercially acceptable confidence interval which is normally ±5%. It is also a higher response rate than achieved in the 2009 survey (61%) and a slightly more representative sample at the sub-group level - a good result.
 
Representativeness of a sample is usually assessed at a 95% confidence level (accuracy) and a ±5% confidence interval (precision).
 
The confidence level tells you how sure you can be. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval. The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain. Most researchers use the 95% confidence level.
 
The confidence interval is the plus-or-minus figure usually reported in newspaper or television opinion poll results. For example, if you use a confidence interval of 4 and 47% of your sample picks an answer, you can be "sure" that if you had asked the question of the entire relevant population between 43% (47-4) and 51% (47+4) would have picked that answer.
 
 
 
All but one of the industry sub-group samples are of sufficient size to be representative of their subgroup populations. There was insufficient number of friendly societies to be statistically representative so results for this sub-group should be treated with caution. They are also the smallest sub-group in the population.
 
Groups are slightly over-represented, but there are sufficient numbers of both group and non-group entities in the response sample for them to be representative of their populations.
 
Smaller entities are slightly under-represented, but the overall sample profile is similar to the population profile — i.e. many more smaller entities than larger entities responded.
 
Overall sample profiles match the population profiles and each sub-group within the sample, apart from friendly societies, are within acceptable confidence intervals - a robust result for a survey of this nature.
 
The response sample profile is displayed in the following tables and charts.
 
INDUSTRY SUB-GROUP POPULATION RESPONSE SAMPLE CONFIDENCE
INTERVAL
  Freq % Freq % ±%
Trustees 217 38.5 119 32.2 6
ADIs 178 31.6 131 35.5 4
General Insurers 123 21.8 84 22.8 6
Life Insurers 30 5.3 28 7.6 5
Friendly Societies 15 2.7 7 1.9 28
Total 563 100.0 369 100.0 3
 
PART OF A GROUP OR NOT POPULATION SAMPLE CONFIDENCE
INTERVAL
  Freq % Freq % ±%
Non-group 425 75.5 255 69.1 4
Group 138 24.5 114 30.9 4
Total 563 100.0 369 100.0 3
 
This bar chart shows the percentage breakdown by population and sample for Industry profile. Click on this image to view the Industry profile in a larger size.
 
Click on the above image to view the Industry profile in a larger size.
 
This bar chart shows the percentage breakdown by population and sample for Group and Non-group profile. Click on this image to view the Group – non-group profile in a larger size.
 
Click on the above image to view the Group – non-group profile in a larger size.
 
Industry and group-relatedness have mixed relationships - see the table below for a cross tabulation with column totals which reflect a split between group and non-group within an industry. Trustees, friendly societies and ADIs are mostly non-group related, while life insurers are most likely to be part of a group. There is not a simple or single pattern between industry and group.
 
  ADI FRIENDLY
SOCIETY
GENERAL
INSURER
LIFE
INSURER
TRUSTEE
n % n % n % n % n %
Group 21 16.0 1 14.3 41 48.8 24 85.7 27 22.7
Non-group 110 84.0 6 85.7 43 51.2 4 14.3 92 77.3
Column total 131 100.0 7 100.0 84 100.0 28 100.0 119 100.0
 
 
Key findings
 
This section outlines the key findings for regulated entities by topic. For each topic the mean scores are presented along with the frequency distribution for each item. The most common themes within free text comments are discussed. Sub-group differences (industry, group affiliation and size) are covered in a later section as are results from knowledgeable observers. In this key findings section, some references are made to changes since 2009, but complete details of year comparisons are presented in a later section.
 
Overall, all topics had a majority of positive responses. Just over 50% of rated items had 100% or more of respondents who agreed or strongly agreed.
 
Prudential requirements
 
APRA's framework
 
The table below displays the mean scores for items related to prudential requirements. On average, respondents agree with APRA's framework and its structure and implementation except for two areas: successful harmonisation and cost. The two lowest scoring items for this topic were also the lowest scoring items in the entire survey as they were in 2009. There has been little change in these scores over time.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
RATED ITEMS –n=369 unless otherwise stated below MEAN
APRA's prudential framework is effective in achieving APRA's mission 4.2
APRA's harmonisation of the prudential framework across its regulated industries is important for your organisation* 4.2
APRA's guidance material is of value to your organisation 4.2
APRA's prudential standards are based on principles rather than detailed prescription 3.9
It is important that APRA closely follows international best practice in making prudential standards for your industry^ 3.8
Prudential standards and guidance material clearly communicate requirements 3.6
Changes to APRA's prudential framework consider the costs of regulation imposed on industry 2.8
APRA has successfully harmonised its prudential framework across the industries it regulates* 2.8
* These items were only asked of entities that are part of a group (n=114)
 
^ This item was asked of all entities except trustees (n=248)
 
The chart on the following page displays the frequency distribution of the rated items for the topic. The majority of items have a majority of positive (agree) ratings. Note the large percentage of no answers for the harmonisation items indicating that a considerable number of respondents did not have a view, possibly because they did not have sufficient knowledge outside their own organisation or group. This no answer percentage is based only on those respondents who were presented the question.
 
This bar chart shows the percentage of responding entities choosing a rating point (n=369 unless stated) for Prudential requirements. Click on this image to view the Prudential requirements in a larger size.
 
Click on the above image to view the Prudential requirements in a larger size.
 
Sources of guidance
 
Regulated entities extensively use all sources of information, with the use of speeches to a lesser extent. All of these percentages have increased slightly since the previous survey which indicates a need or desire to closely monitor APRA announcements and/or decisions. In the chart below, responding entities could choose multiple sources, so totals add to more than 100%.
 
This bar chart shows the percentage of entities choosing a source - multiple answers allowed (% based on n=369) for Guidance sources used. Click on this image to view the Guidance sources used in a larger size.
 
Click on the above image to view the Guidance sources used in a larger size.
 
Those entities that indicated they used their supervisory team as a source of guidance were asked to rate the usefulness of the information from their APRA team. A majority of respondents indicated that the information was useful. See chart below.
 
This pie chart shows the percentage of entities who used a supervisory team (n=319) for Usefulness of guidance from APRA supervisory team. Click on this image to view the Usefulness of guidance from APRA supervisory team in a larger size.
 
Click on the above image to view the Usefulness of guidance from APRA supervisory team in a larger size.
 
Consultation process
 
Mean scores only varied slightly for this topic, with all scores rounding to the agree rating point.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
RATED ITEMS –n=369 unless otherwise stated below MEAN
APRA provides sufficient opportunity for consultation with industry about proposed changes to prudential standards and guidance material 3.9
APRA's consultation packages are readily comprehensible and create a good base for consultation for industry 3.8
APRA considers issues relevant to industry and other stakeholders when developing its prudential standards and guidance material 3.7
 
The chart below displays the frequency distribution of ratings for items within this topic. There are very few negative views about consultation.
 
This bar chart shows the percentage of responding entities choosing a rating point (n=369) for Consultation process. Click on this image to view the Consultation process in a larger size.
 
Click on the above image to view the Consultation process in a larger size.
 
Risk assessments
 
Mean scores for items in this topic were fairly similar. The variation of ratings within items shows that a majority of entities have a fairly similar view and that APRA's risk assessment is working well. However, around 29% of respondents indicate mixed or negative views on APRA's risk assessment compared with their own assessment. As discussed later in the section on predicting overall APRA effectiveness, alignment of risk assessments is a critical item within the whole survey. The group of responding entities with mixed or negative views about risk alignment are most likely to have more negative views of APRA overall.
 
The lower rating for effort required is reflected in the free text comments about the amount of time and resources required to undertake reviews.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
RATED ITEMS –n=369 unless otherwise stated below MEAN
The information collected by APRA in the course of supervision is adequate to assess risks in your organisation 4.0
APRA is effective in identifying risks across your industry in general 3.9
APRA is effective in identifying risks and problems in that part of your organisation that APRA regulates 3.8
APRA's risk assessment of your organisation, as conveyed to you in review reports, is aligned with your organisation's own risk assessment 3.8
APRA's prudential reviews of your organisation are appropriately spaced apart in their timing 3.8
APRA's PAIRS rating reflects your organisation's view of its risk profile 3.6
The effort required of your organisation during APRA's prudential reviews is appropriate 3.5
 
The frequency distribution chart below shows a strong positive bias on most items, but not universally shared views.
 
This bar chart shows the percentage of responding entities choosing a rating point (n=369) for Risk assessments. Click on this image to view the Risk assessments in a larger size.
 
Click on the above image to view the Risk assessments in a larger size.
 
Dealings with APRA
 
APRA staff
 
Respondents, on average, agreed that their APRA teams had good organisational and industry understanding. However, around 20% of entities have mixed or negative views on staff.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
RATED ITEMS –n=369 unless otherwise stated below MEAN
The APRA supervisory team responsible for your organisation has a good understanding of your organisation 4.1
The APRA supervisory team responsible for your organisation is experienced in your industry 4.0
 
The chart below displays the frequency distribution for these items.
 
This bar chart shows the percentage of responding entities choosing a rating point (n=369) for Dealings with APRA. Click on this image to view the Dealings with APRA in a larger size.
 
Click on the above image to view the Dealings with APRA in a larger size.
 
Demonstration of APRA's values
 
APRA staff highly rated for demonstrating the values of integrity and professionalism, and these two items are in the top three rated items of the entire survey. The chart shows that accountability received the highest proportion of negative responses of all values. Note that this topic used a different rating scale compared with most other topics in the survey.
 
Rating scale: 1=never demonstrate, 2=demonstrate to some extent, 3=neutral, 4=demonstrate to a significant extent, 5=always demonstrate
 
RATED ITEMS –n=369 unless otherwise stated below MEAN
Integrity 4.5
Professionalism 4.4
Collaboration 4.1
Accountability 3.9
Foresight 3.8
 
This bar chart shows the percentage of responding entities choosing a rating point (n=369) for Demonstration of APRA values. Click on this image to view the Demonstration of APRA values in a larger size.
 
Click on the above image to view the Demonstration of APRA values in a larger size.
 
Supervisory activities
 
The table below displays the 16 rated items in this topic, sorted from highest to lowest scoring. There is very strong endorsement of single supervisory teams for group entities. On average, respondents agree that APRA effectively enforces its prudential requirements. Consultative and consistent supervision and a principles-focus during visits may need some attention in some areas but this is not so for all entities.
 
Within the free text comments consistency was sometimes raised as an issue for those respondents who have experience of APRA reviewing more than one entity within an industry where the same standard is applied differently. Inconsistency also includes situations where responding entities conduct business with APRA in multiple states, deal with more than one APRA supervision team or interact with different parts of APRA. For this survey, throughout the free text comments there is a small but increasing reference since the previous survey to more prescriptive rather than principle-based practices from APRA.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
RATED ITEMS –n=369 unless otherwise stated below MEAN
A single supervisory team responsible for all group companies is an appropriate way to supervise groups* 4.6
APRA's enforcement of its prudential requirements has had an impact on your industry 4.2
APRA is effective in communicating the findings of supervisory visits to your organisation 4.1
APRA meets its stated approach of being primarily risk-based in its supervision 4.0
APRA's reports of prudential reviews provided to your organisation have the appropriate level of detail 4.0
APRA's supervision practices have had a positive impact on your organisation's risk management practices over the past three years 4.0
APRA's recommendations and suggestions arising from its prudential review of your organisation are useful for your organisation 3.9
During supervisory visits to your organisation, APRA supervisors focus on major risks or controls 3.9
APRA effectively enforces its prudential requirements 3.9
APRA's resolution of your organisation's technical and supervisory requests is satisfactory 3.9
During prudential reviews of your organisation, APRA correctly assesses the importance of issues that are subject to APRA requirements, recommendations or suggestions 3.8
APRA meets its stated approach of supervising in line with international best practice 3.8
APRA meets its stated approach of being forward looking in its supervision 3.8
APRA meets its stated approach of being consultative in its supervision 3.6
APRA meets its stated approach of being consistent in its supervision 3.6
During supervisory visits to your organisation, APRA supervisors focus on principles rather than detailed prescription 3.5
*This item was asked only of group entities (n=114)
 
The frequency distributions for items relating to supervisory activities have been presented as two separate charts on the following page and labelled as charts A and B. Chart A displays the higher scoring items. Note that some of the item wording has been truncated for the charts while full item wording appears in the table above.
 
This bar chart shows the percentage of responding entities who chose a rating point (n=369) for Supervisory activities A. Click on this image to view the Supervisory activities A in a larger size.
 
Click on the above image to view the Supervisory activities A in a larger size.
 
This bar chart shows the percentage of responding entities who chose a rating point (n=369) for Supervisory activities B. Click on this image to view the Supervisory activities B in a larger size.
 
Click on the above image to view the Supervisory activities B in a larger size.
 
Exemptions and variations
 
In total 114 entities (30.9%) had requested an exemption or variation in the past 12 months. Only respondents who had made such a request were asked to rate how well the request was handled.
 
A majority of respondents who had requested an exemption or variation indicated that APRA handled the request well or very well (80.7%) which is an improvement since 2009.
 
HANDLING OF REUEST FOR VARIATION OR EXEMPTION Freq %
Very poorly 2 1.8
Poorly 3 2.6
Neutral 12 10.5
Well 56 49.1
Very well 36 31.6
Don't know / no answer 5 4.4
Total 114 100.0
 
Release of benefits
 
Only 60 entities (16.3%) enquired about early release of superannuation benefits for their members in the past 12 months and 77.9% of respondents indicated APRA handled the request well or very well.
 
HANDLING OF REUEST FOR VARIATION OR EXEMPTION Freq %
Very poorly 0 0.0
Poorly 1 1.7
Neutral 12 20.3
Well 32 54.2
Very well 14 23.7
Total 59 100.0
 
Statistical collections
 
Amount of information
 
More than three-quarters of entities indicated that the amount of statistical data collected by APRA was about right (79.7%), while 19.0% indicated that it was too much or far too much. This is a significant improvement on the 2009 survey results when 66.3% indicated that it was about right and 32.1% indicated that it was too much or far too much.
 
The too much view was supported in some free text comments throughout the survey, and not just comments in this section of the questionnaire.
 
STATISTICAL COLLECTIONS Freq %
Too little 3 0.8
About right 294 79.7
Too much 60 16.3
Far too much 10 2.7
No answer 2 0.5
Total 369 100.0
 
Providing/collecting information
 
The table below displays the mean scores for the rated items in this topic. Instructions on forms and the system to collect statistics are relatively low scoring for this survey. Free text comments support these low ratings, but some respondents have also noted the improvements to the system.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
ITEM MEAN
APRA is helpful when your organisation has difficulties using D2A 3.8
APRA is helpful when your organisation has difficulties understanding APRA's reporting requirements 3.8
The instructions to APRA's statistical forms are helpful 3.6
D2A is easy to use when lodging data with APRA 3.5
 
The chart below displays the frequency distribution of items in the table above.
 
This bar chart shows the percentage of responding entities choosing a rating point (n=369) for Statistical collections. Click on this image to view the Statistical collections in a larger size.
 
Click on the above image to view the Statistical collections in a larger size.
 
Statistical publications
 
Respondents were asked to indicate the various APRA publications their organisation had used in the last 12 months.
 
APRA Insight is the most commonly used publication across all industry sectors, with the periodic bulletins heavily used by insurers of any type.
 
On average, respondents rated the reliability of APRA publications as reliable (mean 4.0) with 79.1% indicating they were reliable or very reliable.
 
This bar chart shows the percentage of entities choosing a source - multiple answers allowed (% based on n=145) for ADI publications used in last 12 months. Click on this image to view the ADI publications used in last 12 months in a larger size.
 
Click on the above image to view the ADI publications used in last 12 months in a larger size.
 
This bar chart shows the percentage of entities choosing a source - multiple answers allowed (% based on n=107) for General insurance publications used in last 12 months. Click on this image to view the General insurance publications used in last 12 months in a larger size.
 
Click on the above image to view the General insurance publications used in last 12 months in a larger size.
 
This bar chart shows the percentage of entities choosing a source - multiple answers allowed (% based on n=138) for Superannuation publications used in last 12 months. Click on this image to view the Superannuation publications used in last 12 months in a larger size.
 
Click on the above image to view the Superannuation publications used in last 12 months in a larger size.
 
This bar chart shows the percentage of entities choosing a source - multiple answers allowed (% based on n=167) for Life insurance and friendly society publications used in last 12 months. Click on this image to view the Life insurance and friendly society in a larger size.
 
Click on the above image to view the Life insurance and friendly society publications used in last 12 months in a larger size.
 
 
Predicting effective enforcement
 
ASR conducted step wise linear multiple regression and used the item APRA effectively enforces its prudential requirements as the dependent variable.
 
All other items were initially used as independent variables in the regression, and through iterative regression passes using the step wise method, insignificant independent variables were dropped, leaving only significant items as possible predictors of APRA enforcing its prudential requirements.
 
The overall model was then checked for significance using an ANOVA test and the extent to which the chosen significant independent variables could explain the dependent variable (R squared).
 
Close to 60% of all the variance in the score for the dependent variable can be explained by two items, determined as having highest significance in the model:
  • APRA's risk assessment of your organisation, as conveyed to you in review reports, is aligned with your organisation's own risk assessment (mean 3.8)
  • APRA meets its stated approach of supervising in line with international best practice (mean 3.8).
Close to 70% of all variance in the score for the dependent variable can be explained by four items - the above two plus:
  • APRA's supervision practices have had a positive impact on your organisation's risk management practices over the past three years (mean 4.0)
  • APRA's guidance material is of value to your organisation (mean 4.2).
The four items are presented in decreasing order of impact, so alignment of APRA and organisational risk assessment has the greatest impact.
 
These are the four items which APRA needs to focus on in order to be perceived as effectively enforcing its requirements. It is interesting to note that these are not the highest scoring items in the survey and they are far from the lowest. Where entities have a realistic understanding of their risk profile (as in aligned with APRA's view of their risk profile) and where APRA reviews have created positive change, there is likely to be an overall positive view of the organisation.
 
This predictive view of enforcement also works in reverse, meaning that where entities have negative views of APRA there is more likely to be mis-alignment between the entity's and APRA's views of the organisation's risk profile, APRA is seen to be not supervising in line with international b est practice and guidance material has been of less value, etc.
 
The predictive items are simply offered as a way of helping identify where initiatives should be maintained or where improvements could be focused.
 
 
Industry comparison
 
When providing the respondent file, APRA allocated regulated entities to one of five industry sub-groups. This section compares the results for these sub-groups. However, because one of the groups, friendly societies, is so small this group has been dropped when calculating statistical difference. Comparing a sample of seven with one of 131 can produce unreliable and, in some cases, meaningless statistical results. Also note that the cell counts (the number of respondents that means are calculated on) for friendly societies in some cases are extremely small.
 
Cautions
 
When interpreting statistical significance, caution must be taken when comparative sub-groups have very different sizes. Large difference in sample sizes can affect the results of some statistical tests.
 
It is important to understand that statistically significant differences reflect underlying variations in scores rather than a difference in absolute amounts. For example, in some instances a difference of a tenth of a rating point in a mean score may be statistically significant, while a difference of a whole rating point between two mean scores may not be. So while some scores are quite different in absolute terms they may not be statistically significantly different and vice versa. Rounding also tends to add to what might seem a nonsensical result. Two items can have the same mean when rounded to a single decimal place, but they can still be significantly different.
 
The industry breakdown of respondents is displayed in the table below.
 
INDUSTRY Freq %
ADI 131 35.5
Trustee 119 32.2
General insurer 84 22.8
Life insurer 28 7.6
Friendly society 7 1.9
Total 369 100.0
 
Statistically significant industry differences
 
Of the 45 items that were rated using a five-point scale, 33 were statistically different at the 95% confidence level. There are considerable differences between industries but there is little pattern to the differences. For example in 2009, ADIs tended to be more positive whereas in 2011 ADIs are more positive on some items, and more negative on other items. This result indicates that particular issues or activities in the past two years have affected different industry groups, or the same issue has affected industries differently. So segments within the overall regulated entity population have quite different views about much of APRA's activities. They are not as homogenous as in 2009.
 
Legend: Blue bold indicates significantly higher (more positive) than the other groups, while yellow italics indicates significantly lower than the other two groups (more negative).
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
STATISTICALLY SIGNFICANTLY DIFFERENT ITEMS ADI
n=131
GENERAL INSURER
n=84
LIFE INSURER
n=28
TRUSTEE
n=119
APRA’s prudential framework is effective in achieving APRA’s mission 4.2 4.3 4.1 4.0
APRA’s prudential standards are based on principles rather than detailed prescription 4.0 3.7 3.5 4.0
Prudential standards and guidance material clearly communicate requirements 3.5 3.7 3.5 3.8
APRA’s harmonisation of the prudential framework across its regulated industries is important for your organisation 4.3 3.5 4.4 4.7
APRA has successfully harmonised its prudential framework across the industries it regulates 3.0 2.8 2.7 2.8
APRA’s guidance material is of value to your organisation 4.1 4.0 4.0 4.3
Changes to APRA’s prudential framework consider the costs of regulation imposed on industry 2.7 2.8 2.7 3.0
APRA considers issues relevant to industry and other stakeholders when developing its prudential standards and guidance material 3.6 3.7 3.6 3.8
APRA’s consultation packages are readily comprehensible and create a good base for consultation for industry 3.7 3.7 4.0 3.8
APRA is effective in identifying risks and problems in that part of your organisation that APRA regulates 3.9 3.7 4.0 3.9
APRA’s risk assessment of your organisation, as conveyed to you in review reports, is aligned with your organisations own risk assessment 3.6 3.9 4.0 3.8
APRA’s PAIRS rating reflects your organisations view of its risk profile 3.5 3.8 3.5 3.6
The effort required of your organisation during APRA’s prudential reviews is appropriate 3.6 3.4 3.0 3.5
The APRA supervisory team responsible for your organisation is experienced in your industry 4.0 3.8 3.8 4.1
APRA staff’s demonstration of integrity 4.6 4.6 4.6 4.4
APRA staff’s demonstration of collaboration 4.2 4.3 4.3 3.9
APRA staff’s demonstration of professionalism 4.5 4.4 4.5 4.2
APRA staff’s demonstration of foresight 3.9 4.1 3.9 3.6
APRA staff’s demonstration of accountability 4.0 3.9 4.1 3.8
APRA meets its stated approach of being forward looking in its supervision 3.8 3.9 3.7 3.6
APRA meets its stated approach of being primarily risk-based in its supervision 4.2 3.9 3.9 4.0
APRA meets its stated approach of being consultative in its supervision 3.8 4.0 4.0 3.9
APRA meets its stated approach of being consistent in its supervision 3.5 3.6 3.7 3.7
APRA meets its stated approach of supervising in line with international best practice 3.9 3.8 4.0 3.6
During supervisory visits to your organisation, APRA supervisors focus on principles rather than detailed prescription 3.6 3.3 3.2 3.6
APRA is effective in communicating the findings of supervisory visits to your organisation 4.0 4.2 4.1 4.1
APRA’s resolution of your organisations technical and supervisory requests is satisfactory 3.8 3.8 4.0 3.9
APRA effectively enforces its prudential requirements 4.2 4.2 4.1 4.0
APRA’s enforcement of its prudential requirements has had an impact on your industry 4.3 4.3 4.3 4.1
APRA’s supervision practices have had a positive impact on your organisations risk management practices over the past three years 4.0 3.8 3.9 4.0
APRA is helpful when your organisation has difficulties understanding APRA’s reporting requirements 3.9 3.9 3.6 3.6
APRA is helpful when your organisation has difficulties using D2A 4.0 3.8 3.8 3.6
Reliability of the APRA publications your organisation uses* 4.2 4.0 3.9 4.0
* based on rating scale of 1=very unreliable, 2=unreliable, 3=neutral, 4=reliable, 5=very reliable
 
Prudential requirements
 
The table below displays all rated items in the topic by industry sub-group. There are some mixed results. Harmonisation across industries is considerably more important for life insurers than for other industries. Insurers and trustees rate the success of harmonisation lower than other industry groups while trustees have a slightly more positive view of the impact of regulatory costs on the industry.
 
This bar chart shows the mean industry scores for Prudential requirements. Click on this image to view the Prudential requirements in a larger size.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
* Sample too small to include result for friendly societies
 
^ This item was asked of all entities except trustees (n=248)
 
Click on the above image to view the Prudential requirements in a larger size.
 
Consultation process
 
There are some differences around consultation, with friendly societies more positive on two of the three items while ADIs and life insurers were less happy about APRA's consideration of industry issues when developing prudential standards. Free text comments indicate that consultation should be broader and should bring small as well as large entities to the table for discussion.
 
This bar chart shows the mean industry scores for Consultation process. Click on this image to view the Consultation process in a larger size.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
Click on the above image to view the Consultation process in a larger size.
 
Risk assessments
 
Within the topic of risk assessments there are some differences across industries, particularly in relation to the amount of effort required during reviews. Life insurers take a much more negative view on this item which was the same result in 2009.
 
This bar chart shows the mean industry scores for Risk assessments. Click on this image to view the Risk assessments in a larger size.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
Click on the above image to view the Risk assessments in a larger size.
 
Dealings with APRA
 
Understanding and experience
 
Similarly, when rating APRA staff's understanding and experience there was little variation between industries with insurers slightly less positive about their APRA supervisory team's industry experience. All items rated around the agree rating point.
 
This bar chart shows the mean industry scores for Dealings with APRA. Click on this image to view the Dealings with APRA in a larger size.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
Click on the above image to view the Dealings with APRA in a larger size.
 
Demonstration of APRA's values
 
APRA staff's demonstration of values was rated highly across all industries with the exception of friendly societies. Note that this group has a very small sample size, so one or two very negative views will affect the overall result considerably. As a pattern, trustees also have lower ratings of APRA staff's demonstration of values than ADIs and insurers.
 
This bar chart shows the mean industry scores for Demonstration of APRA's values. Click on this image to view the Demonstration of APRA's values in a larger size.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
Click on the above image to view the Demonstration of APRA's values in a larger size.
 
Supervisory activities
 
In terms of differences, ADIs tend to be more negative about consistency in supervision while life insurers and friendly societies are more negative about APRA's focus on principles rather than prescription. Friendly societies are also more negative about alignment with international best practice, probably because this is of less concern to them.
 
This bar chart shows the mean industry scores for Supervisory activities chart A. Click on this image to view the Supervisory activities chart A in a larger size.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
Click on the above image to view the Supervisory activities chart A in a larger size.
 
This bar chart shows the mean industry scores for Supervisory activities chart B. Click on this image to view the Supervisory activities chart B in a larger size.
 
* This item was only asked of respondents who were part of a group
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
Click on the above image to view the Supervisory activities chart B in a larger size.
 
Statistical collections and publication reliability
 
Most items in this topic scored similarly across industries. However, friendly societies have more negative views about D2A which they also did in 2009. There are few differences in reliability ratings with ADIs scoring most positively.
 
This bar chart shows the mean industry scores for Statiscal collections and publication reliability. Click on this image to view the Statistical collections and publication reliability in a larger size.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree while * used 5 point scale of very unreliable to very reliable
 
Click on the above image to view the Statistical collections and publication reliability in a larger size.
 
 
Group comparison
 
This section discusses only those items that were statistically significant different between group and non-group entities and where both group and non-group entities answered the same questions. Overall there are not many differences and items of difference cover a broad range of topics. Where there are differences, generally, group entities rate items higher or more positively than non-group entities but this does not appear to be a reflection of organisational size, as group entities rate the appropriateness of effort required lower than non-groups. This item about appropriateness of effort has the largest difference in mean scores between the two categories (group and non-group).
 
Items that are statistically significant different tend to focus on staff, interacting with APRA and supervision.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree while * items used a 5 point never-always scale
 
STATISTICALLY SIGNFICANTLY DIFFERENT ITEMS GROUP
n=114
NON-GROUP
n=255
APRA’s prudential standards are based on principles rather than detailed prescription 3.7 4.0
It is important to your organisation that APRA closely follows international best practice in making prudential standards for your industry 4.0 3.7
Usefulness of the information your organisation receives from your APRA supervision team (only answered by those with supervision teams) 2.8 2.6
APRA’s consultation packages are readily comprehensible and create a good base for consultation for industry 3.9 3.7
APRA is effective in identifying risks across your industry in general 4.1 3.9
APRA’s risk assessment of your organisation, as conveyed to you in review reports, is aligned with your organisations own risk assessment 3.9 3.7
The effort required of your organisation during APRA’s prudential reviews is appropriate 3.2 3.6
APRA staff’s demonstration of integrity* 4.7 4.4
APRA staff’s demonstration of professionalism* 4.5 4.3
APRA staff’s demonstration of accountability* 4.1 3.9
APRA meets its stated approach of being primarily risk-based in its supervision 3.9 4.1
APRA meets its stated approach of being consultative in its supervision 4.1 3.8
APRA meets its stated approach of being consistent in its supervision 3.8 3.5
APRA meets its stated approach of supervising in line with international best practice 4.0 3.7
APRA is effective in communicating the findings of supervisory visits to your organisation 4.2 4.0
 
 
Year comparison
 
Despite changes in the number of entities involved in the 2009 and 2011 surveys and changes to the financial services industry, there have been very few significant changes to the Stakeholder Survey results. Only five of the 45 rated items compared were statistically significantly different. All other results between the two years were the same within the ±5% confidence interval. This indicates a very stable pattern of behaviour from APRA and few changes in stakeholders' perceptions of or views about APRA in the past two years.
 
The table below displays the significantly different items.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree
 
STATISTICALLY SIGNFICANTLY DIFFERENT ITEMS 2009
n=392
2011
n=369
APRA has successfully harmonised its prudential framework across the industries it regulates 3.4 2.8
During supervisory visits to your organisation, APRA supervisors focus on principles rather than detailed prescription 3.6 3.5
During supervisory visits to your organisation, APRA supervisors focus on major risks or controls 4.0 3.9
A single supervisory team responsible for all group companies is an appropriate way to supervise groups 4.3 4.6
APRA is helpful when your organisation has difficulties using D2A 3.9 3.8
 
 
Knowledgeable observers
 
Overview
 
A range of knowledgeable observers were invited to participate in the Stakeholder Survey, including industry representatives as well as appointed actuaries and auditors on APRA's contacts database. In 2009, 30 industry representatives were included in the survey and only 17 people responded. To increase the response base APRA decided to include in the 2011 Survey some appointed actuaries and auditors who were familiar with APRA and how it operates.
 
APRA provided ASR with a list of 160 knowledgeable observers. Some were from the same organisation and some of these people chose a designated person to answer once from that organisation, while others were not available to answer during the survey period or no longer working in Australia and/or in the industry.
 
Because of the way in which knowledgeable observers were selected and the ways in which they chose to answer/not answer, it is difficult to obtain an accurate count of the total knowledgeable observer population in Australia. It is more important that 61 people responded and that this is considerably more than the 17 who responded in 2009. So the 2011 response set includes a much broader range of views.
 
Note that three respondents answered as both a knowledgeable observer and a regulated entity contact. These people were clear about the different perspectives they had to answer from prior to answering each survey.
 
The Knowledgeable Observer Survey was based on the regulated entities' Survey but much shorter. The questions were changed slightly to reflect a broader perspective than a single organisation. Because of the different target population between the 2009 and 2011 surveys, a year comparison was not conducted.
 
Comparison with regulated entities
 
A majority of the comparable items are statistically significantly different and knowledgeable observers tend to rate APRA higher than regulated entities. The exception is around APRA staff's demonstration of APRA's values. Knowledgeable observers rated APRA significantly lower than regulated entities on all values except integrity.
 
There may be many reasons for the statistically significant differences, but some likely explanations are that knowledgeable observers have a broader and deeper understanding of the industry as a whole and more understanding of APRA's rationale for making decisions. However, they may also have less to do with APRA's supervisory staff on an operational level.
 
Knowledgeable observers endorse the regulatory framework and its implementation as well as APRA's guidance material and its consultative practices.
 
Scale legend: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree Green indicates item is statistically significantly different between two groups
 
COMPARABLE ITEMS KNOWLEDGEABLE OBSERVERS
n=61
REGULATED ENTITIES
n=369
APRA staff’s demonstration of integrity* 4.5 4.5
APRA's enforcement of its prudential requirements has had an impact on regulated institutions 4.4 4.2
APRA's prudential framework is effective in achieving APRA's mission 4.3 4.2
APRA's guidance material is of value to your organisation 4.2 4.2
APRA provides sufficient opportunity for consultation with industry about proposed changes to prudential standards and guidance material 4.2 3.9
APRA staff’s demonstration of professionalism* 4.2 4.4
APRA effectively enforces its prudential requirements 4.2 4.1
APRA meets its stated approach of being primarily risk-based in its supervision 4.2 4.0
APRA's consultation packages are readily comprehensible and create a good base for consultation for industry 4.1 3.8
APRA meets its stated approach of supervising in line with international best practice 4.0 3.8
APRA considers issues relevant to industry and other stakeholders when developing its prudential standards and guidance material 4.0 3.7
Reliability of the APRA publications your organisation uses^ 3.9 4.0
A single supervisory team responsible for all group companies is an appropriate way to supervise groups 3.9 4.6
Prudential standards and guidance material clearly communicate requirements 3.9 3.6
APRA meets its stated approach of being consultative in its supervision 3.9 3.9
APRA meets its stated approach of being forward looking in its supervision 3.8 3.8
APRA's prudential standards are based on principles rather than detailed prescription 3.8 3.9
APRA staff’s demonstration of collaboration* 3.7 4.1
APRA staff’s demonstration of accountability* 3.6 3.9
F APRA staff’s demonstration of foresight* 3.5 3.8
APRA meets its stated approach of being consistent in its supervision 3.5 3.6
Changes to APRA's prudential framework consider the costs of regulation imposed on APRA regulated institutions 2.9 2.8
* Based on 5 point never-always scale
 
^ Based 5 point very unreliable to very reliable scale
 
In terms of information sources, knowledgeable observers use APRA's general website information and prudential practice guides the most. This pattern is quite different to regulated entities with the exception of practice guides which are heavily used by both.
 
This bar chart shows the percentage of entities choosing a source - multiple answers allowed (% based on n=61) for Guidance sources used by knowledgeable observers. Click on this image to view the Guidance source used by knowledgeable observers in a larger size.
 
Click on the above image to view the Guidance source used by knowledgeable observers in a larger size.
 
Knowledgeable observers were asked to indicate the APRA publications their organisation had used in the last 12 months. APRA Insight was used most commonly for all industries.
 
This bar chart shows the percentage of respondents choosing a source - multiple answers allowed (% based on n=61) for ADI publications used in the last 12 months. Click on this image to view the ADI publications used in last 12 months in a larger size.
 
Click on the above image to view the ADI publications used in last 12 months in a larger size.
 
This bar chart shows the percentage of respondents choosing a source - multiple answers allowed (% based on n=61) for General insurance publications used in the last 12 months. Click on this image to view the General insurance publications in a larger size.
 
Click on the above image to view the General insurance publications used in last 12 months in a larger size.
 
This bar chart shows the percentage of respondents choosing a source - multiple answers allowed (% based on n=61) for Superannuation publications used in the last 12 months. Click on this image to view the Superannuation publications in a larger size.
 
Click on the above image to view the Superannuation publications used in last 12 months in a larger size.
 
This bar chart shows the percentage of respondents choosing a source - multiple answers allowed (% based on n=61) for life insurance and friendly society publications used in the last 12 months. Click on this image to view the Superannuation publications in a larger size.
 
Click on the above image to view the life insurance and friendly society publications used in the last 12 months in a larger size.
 
 
Conclusions
  1. Overall the results of this survey are positive for APRA, particularly as in most areas there have been few changes since 2009. Regulated entities and knowledgeable observers support APRA's framework and regulatory approach of principles-based supervision. A majority of respondents agree that APRA effectively enforces its prudential requirements and believe that APRA has had a positive impact on their industry.
  2. APRA's strengths are its staff's integrity and professionalism, as well as the positive impact the enforcement of prudential requirements has had on the industry and APRA's guidance material.
  3. Areas scoring lowest and which may benefit from attention are consideration of the cost of regulation, harmonisation across regulatory authorities and across both national and international standards.
  4. There are considerable differences in views between industry groups which indicate that some changes have been made that have affected industry groups differentially. However, there is no pattern to these differences: one industry group is not universally more negative or positive.
  5. There are few changes in numeric results since 2009 which indicates that APRA's behaviour has been fairly stable and/or that regulated entities are hard to shift in their views.