How Regulators Can Improve Their Effectiveness

Share

Regulator Effectiveness

This post suggests an approach for regulators to improve cost-effectiveness by focusing on high-risk areas to protect people whilst minimising burden.

Regulation can have a positive effect in protecting us (as workers, individuals, consumers, patients, etc.), but it may also have a negative effect in terms of the burdens imposed on the businesses and professionals who comply with those regulations.

A big issue for regulators is how to maximise the positive effects of protection whilst minimising the negative effects of burden. If either the positive or the negative effects are unbalanced, then the impact will be disproportionate with either too little protection or too much cost to businesses and professionals.

This issue has been made more difficult by the budget reductions that many regulators have faced in recent times. However, these budget reductions also reinforce the need for regulators to focus their resources on the key risks and to be cost effective.

My experience of working with regulators over the last 15 years has given me insight into the approaches that have been effective for those regulators. I have combined a range of experiences to suggest an eight-point approach for intervening with an industry or sector. This approach comprises the following eight points, and allows regulators to focus their resources cost effectively on the high-risk areas:

  1. Decide what outcomes you want
  2. Understand what the risks are
  3. Use your data to identify the high-risk areas
  4. Understand the people and organisations you regulate and what influences them
  5. Identify potential options for intervention
  6. Prioritise the intervention options identified
  7. Identify how the selected interventions are likely to work and what constitutes success
  8. Evaluate the interventions

The approach requires regulators to identify the high-risk areas and the types of people and organisations associated with those areas. Once identified, the regulator needs to understand what influences these people and organisations and segment them. In this way, the regulator can identify and prioritise a set of interventions that focus on increasing compliance in the high-risk areas.

Some established regulators will already be taking an approach similar to the one described here. In that case, this article may help to confirm that they are heading in the right direction, and perhaps identify a few additional points to consider when they next refresh their intervention strategy. For newer regulators, this article may help them in learning from the experience of others and providing a useful starting point or checklist.

1. Decide what outcomes you want

This may sound like an obvious point, but it can be too easy to focus on what you are doing rather than what you should be doing. If a regulator focuses on the number of inspections or number of prosecutions, it may be missing out on significant opportunities to achieve compliance among the people and organisations that it regulates.

The number of inspections gives an easy measure of activity. Inspections can have an impact on those inspected. However, there are limitations to inspection; not least, the number of inspections a regulator can do when budgets are reducing, the limited proportion of the population they reach and their effectiveness when not focussed on high-risk areas.

The number of prosecutions is also easy to measure. Prosecutions also have limitations in that they tend to take several years to get to court, few prosecutions are taken and they are ‘after the event’.

If regulations are designed to improve risk management, reduce risk and protect people, then achieving compliance with those regulations should be the key outcome for a regulator. Regulators can use a range of activities (typically known as interventions) to help people and organisations comply with the relevant regulations.

The rest of this article focuses on an approach regulators can use to select a cost-effective set of interventions to maximise compliance rates.

2. Understand what the risks are

In most industries and sectors there will be a range of potential risks ranging from minimal to severe. If a regulator does not focus on the most significant risks, this may lead to too little protection and too much cost to low risk businesses and professionals.

The risks can typically be identified by experienced people. A workshop with input from both industry and regulators is an effective way to identify risks. The output should be a structured list of risks. However, this list should not just contain historical risks; it should contain potential risks that participants can see coming in the future as a result of changes in practices, economics or other regulations.

3. Use your data to identify the high-risk areas

Two issues need to be addressed:

  • What are the most significant risks – where are the most severe risks occurring and how many are there
  • Which people or organisations are associated with the most significant risks – are small or large organisations involved, how long have they been operating, where are they based, what do they do

Without this information, it will not be possible for a regulator to focus its resources cost effectively.

There are typically, four sources of data for addressing these two issues:

  • Use of a regulator’s own collected data
  • Review of past incidents
  • Views of experienced people
  • Use of third party data (e.g. insurers)

In my experience, the first two sources provide the most comprehensive picture. Regulator’s data provide good insight into the quantitative issues (how many, what type, where, what was involved), whilst a review of past incidents provides insight into qualitative issues (what happened and why). However, this does rely on regulators collecting good quality data – there is a difference between the quality of data required for annual reports and that required for proactive targeting of resources. Where available, the views of experienced people are very useful for adding to that qualitative insight; in particular, to assess whether historical data is compatible with future practice.

Some regulators use risk rating tools. In these tools, the regulator’s data was used to develop profiles of the organisations they regulate. These tools allow those that pose the most significant risks to be identified and resources focused in their direction.

Whilst third party data is potentially useful, data sharing protocols, commercial sensitivities and confidentiality issues often prevent regulators getting access to data in a detailed format.

4. Understand the people and organisations you regulate and what influences them

Compliance does not come naturally to all people and organisations, and regulators need to find ways to get those people and organisations to change their attitudes and behaviours. Getting people and organisations to change their behaviours is not easy. If regulators do not understand the people and organisations they regulate, behaviour change is near impossible.

There are three key issues for regulators:

  • The sector demographics – understand how the sector is made up in terms of size, location, type of work, profitability (a small number of large organisations is somewhat easier to intervene with and influence than a large number of smaller organisations)
  • Segment the sector – identify the common factors that link groups of organisations or people (each will require a different approach to get the compliance message over)
  • The influences on that sector and the critical influence paths – understand if they are they influenced by regulators, their peers, customers, etc. and how that influence gets transmitted through an organisation (I have found the Influence Network to be an effective tool for doing this)

By understanding these issues, regulators will be better able to tailor their interventions to the critical influence paths which are likely to be the most effective routes to changing behaviour (e.g. interventions with the head office of large organisations to get the message throughout the organisation). The MINDSPACE report provides useful material for regulators on influencing behaviour.

5. Identify potential options for intervention

There are numerous ways that regulators can intervene over and above the well-known approaches of guidance, inspection and prosecution. However, many regulators have had their budgets reduced, so they need to focus their resources on those interventions that will give the best return on investment.

There are three points when a regulator can intervene:

  • Before the breach of the regulations e.g. partnership working, motivating senior managers, using the supply chain, education, awareness raising
  • During the breach e.g. inspections, high regulator visibility, statutory notices
  • After the breach e.g. investigations, complaints, prosecutions

In my experience, the best way to get a reasonable list of interventions is to hold a workshop with experienced regulatory staff and get them to suggest interventions under these three headings. At this point, the important thing is to generate plenty of options – we can prioritise them in the next step.

To help generate relevant suggestions, I suggest that participants:

  • Do not limit themselves to what they have used in past – look at other regulators and also anticipate the changes that are coming
  • Identify who else they can work with – there are likely to be partner organisations that can share the workload and increase the credibility of the regulator’s interventions

If a regulator is focussing on achieving compliance, then a large proportion of its interventions should be focussed on people and organisations before they breach the regulations. Regulators should also think about how those interventions made during and after the breach can be used to spread the message about compliance to a wider audience (e.g. publicising the lessons learned so that others are more likely to comply).

6. Prioritise the intervention options identified

A long list of potential interventions minimises the risk that good ideas may have been missed. However, a regulator can only implement a subset of those interventions. In my opinion, those implemented should be the ones that provide the best return on investment – they are easy to implement and have significant impact.

In my experience, a workshop using an option appraisal tool is the best ways to get a prioritised set of interventions.

Before any prioritisation takes place though, we need to agree on the criteria against which each intervention will be rated. These will vary depending on the particular regulator, but my suggestions for an initial set of criteria are:

  • Ease of intervention – cost or effort, industry partners, track record (new or existing), ease of evaluation
  • Impact of intervention – number of people or organisations affected, focus of the intervention, severity of the risk, fit with regulator’s priorities
  • Timing of intervention in relation to the risk – before, during or after
  • Time taken to have an impact – immediate, intermediate or long term
  • Time that the impact lasts for – short, medium or long term

It may take several iterations to reach agreement, but the result will be a prioritised set of interventions supported by an audit trail. An example of how this approach was used is shown Section 13 of HSE Research Report 356.

7. Identify how the selected interventions are likely to work and what constitutes success

Once a set of interventions has been identified, the next step is to determine how those interventions are likely to work so that they can be implemented successfully. Without this step, there is the risk that those interventions will not be focussed properly or implemented effectively, or we may not know whether they are working or not.

In an ideal world, interventions would have an immediate effect on duty holders, and people would change their behaviour ‘overnight’. However, it takes time for the effects of interventions to have an impact and deliver outcomes. The impact will take time as the target audience needs to progress from awareness to attitude change to behaviour change before tangible results are apparent.

As we do not live in an ideal world, an approach is required to:

  • Describe how and why the interventions are likely to work
  • Link the inputs made by the regulator, to the impact and outcomes

A theory of change provides the former, whilst a logic model provides the latter. With these tools we can identify the steps needed to implement the interventions along with indicators that will provide evidence on whether the interventions have met their objectives and delivered outcomes in the short, medium and long term.

8. Evaluate the interventions

The effective regulator’s work does not end when the interventions are implemented. It is an iterative process, and we need to evaluate whether those interventions are working or not – even in the short term when little data are available. Not all interventions will work, and some may not work as well as they did previously. If the regulator does not evaluate the effectiveness of their interventions, then time, money and resources may be wasted on interventions that are not working.

The effective regulator will monitor its interventions and get feedback so that changes can be made if necessary. This need not be an expensive exercise – evaluation just needs to be proportionate. The indicators identified in the previous step could be based on information that is routinely collected by the regulator or partner organisations that may be willing to share non-sensitive information.

In conclusion

These points provide an overview of how regulators can focus their resources cost-effectively.

The approach requires regulators to identify the high-risk areas and the types of people and organisations associated with those areas. Once identified, the regulator needs to understand what influences these people and organisations and segment them. In this way, the regulator can identify and prioritise a set of interventions that focus on increasing compliance in the high-risk areas.

The approach aims to help regulators focus on the high-risk areas so that people are protected from the higher risks whilst businesses and professionals are not unduly burdened.


About the author:

Mike Webster specialises in risk and regulation, and is a chartered engineer with over 25 years’ experience. He founded MPW R&R to provide risk and regulatory solutions to regulators, regulated organisations and litigants in safety critical industries.

For more information and advice, sign up for regular updates or drop me a line at mike.webster@mpwrandr.co.uk


 

Share

Leave a Reply

Your email address will not be published. Required fields are marked *

*