Item 6
GCP(13)24
Outcome performance measures
For Board approval For Board briefing For Board steer
For Board information
Prepared by:
Rebekah Eden
Programme director evidence and analysis
Executive summary
1. This paper updates the Board on progress towards the development of the Gambling Commission’s (the Commission) corporate outcome performance measures. We seek endorsement on the proposed framework, next steps and any comments/suggestions. 2. At the Board Strategy away day, the Board identified outcomes of the Commission’s
activity following a visioning session (session 1). To achieve this, the Board were asked to visualise what the Commission should look like in three years time. The Board also proposed draft indicators (session 2). Following both sessions, a steer was given for the production of a note of the session (circulated previously), and a high level outline of a performance measurement framework.
3. Using this as a foundation, a draft framework has been developed for consideration by Board. The framework is accompanied by an example, using the workstream Anti Money Laundering to demonstrate how deliverables from workstreams feed into the desired outcomes, and to illustrate how indicators/measures from the workstreams will be used to form intermediate or proxy measures for the Commission’s performance in using three ‘levers’ the Board identified.
4. We ask that the Board provide a steer on the draft framework. A more detailed framework and measures will be shared with the Board at its meeting in September.
Background
5. The Board considered, as part of its strategy day, a paper on the development of strategic performance measures. It was agreed that identified performance measures should seek to be both diagnostic and provide accountability. Such measures would help the Commission to improve its effectiveness whilst having the capability of being used to hold the Commission account.
6. At the Board Strategy away day, Board members were asked to develop a three-year vision for the Commission i.e. what Commission was aiming for in terms of outcomes in three years time in line with our statutory duties to permit gambling subject to the licensing objectives and to provide advice. The Board agreed the following outcomes:
The gambling industry has a culture which supports a safe leisure activity, providing transparency for consumers and,
Public policy and debate on gambling is based on expert advice for Government, public and other stakeholders.
7. A further discussion took place on how the Commission would identify when it had reached the vision (session 2 of the board away day). Three key areas (or levers) where the Commission might focus its attention to monitor and ensure progress towards the vision were identified as:
Indications of whether the industry was taking responsibility for the licensing objectives, ensuring gambling is being provided safely and in a transparent way, helping to reduce the perception of gambling being related to crime as opposed to it being a recreational activity, (industry taking responsibility)
Use of hard enforcement powers and softer including educational approaches as preventative measures, providing regulation that is efficient and effective, (hard/soft approaches)
The Commission is more ‘intelligent’ and has a strong evidence base, which it builds on, continually scanning and searching, exchanging information with others with expertise and experience including international partners and academics. (rigorous information)
8. Against each of the outcomes potential indicators and measures were identified. Table 1: Identified outcomes, indicators and potential measures
Outcome Indicator Measure
Industry taking responsibility
Evidence that the industry actively engages in forums for change
Sharing best practice
Standard code of conduct
Improvement in compliance demonstrated through the assessment matrix
Changes in flow of enhanced compliance cases
Clarity in annual reports demonstrating operators are engaged and driving the desired culture
Industry collecting and using data to demonstrate their responsibility for the licensing objectives
Level of positive feedback from stakeholders
# best practice documents developed and shared
# standard code of conducts developed by operators Level of change in compliance through assessment matrix # enhanced compliance cases
# Industry annual reports demonstrating responsibility for licensing objectives (focus on high impact operators?)
Level of information sharing between operator and Commission
Hard/soft approaches
Routine cases
precedent setting cases
Level of publicity/education where feedback states if warns or deters
Impact of enhanced compliance cases
Board review of relevant issues
# routine cases
# Enhanced compliance
Level of publicity regarding gambling regulation
Level of educational activities
Feedback from stakeholders
Rigorous information
Commission reliable, credible source of gambling statistics
Commission produces good quality research on relevant
Level of stats publications which are used by
stakeholders
Outcome Indicator Measure issues
Meaningful exchange of information internationally
Horizon scanning with operators for assessing risk
Exchange of secure information with law enforcement agencies
stats/research publications
Level of engagement in the international landscape e.g. invites to speak at
conferences, engagement in international forums
Feedback from sector
Level of information exchange
9. Collaboration/engagement and the Commission building its reputation as an authority on gambling regulation were identified as important elements in achieving the three
outcomes identified above. Whilst neither are outcomes in their own right, they were recognised as important elements in achieving the outcomes and ones which run systematically through most of all we do.
10. Following the Board Strategy away day, a short note of sessions 1 and 2 of the day was circulated to colleagues who attended the day. Comments were received to note the importance of data collection which might also include data from any foresight exercises e.g. three-five year scenario forecasting jointly with the industry. It was also noted that no consideration was given to the National Lottery Commission (NLC) merger and future consideration of measures to inform progression towards outcomes. These points will be taken into account in taking this work forward. In particular we are discussing with NLC colleagues how best to incorporate their additional objective into this framework. 11. It was agreed with Board that an early draft framework would be developed for
consideration at its meeting in May. The framework is set out below.
Issues
The Framework
12. The purpose of this Performance Measurement Framework is to provide a consistent approach for systematically collecting, analysing, utilising and reporting on the performance of the Commission’s programme of activity. This framework is a management tool that will enhance the management and reporting of the Business plan’s programmes and activities by measuring the Commission’s level of achievement of results. It will enable colleagues to make more informed and effective choices and decisions about their programmes and activities.
13. Based on discussions to date, we have identified the requirement for the Commission to articulate the impacts of its work. This starts with having a clear sense of direction (top down approach), communicating how the operational activity informs this (bottom up). This is set out in figure 1 below.
14. The vision sets out the goals of the Commission within three years, and the workstream activity identified within the business plan sets out the plan for how the Commission will attempt to reach the goals. This involves a two way flow of information with the vision driving activity and the activity informing the vision on progress.
15. Progress against workstream activity, in the first instance (2), will be monitored through management information. These will focus on tangible deliverables (the outputs of the activity), and provide programme leads, workstream leads with a sense of workstream progression and how much each activity has delivered. In other words, how much we have done.
16. All workstreams have identified outcomes of its activity. These are identified as
intermediate outcomes, as they are still removed from the 3 outcomes/levers identified at the Board Strategy away day (although in principle all workstreams should be directly and explicitly linked to our statutory duties. This is somewhat sketchy in many cases.). This moves performance measurement into the space of answering how well the
Commission has delivered (3). Again programme leads and workstream leads together, will identify how well the intermediate outcomes of the workstream align with proxy outcome measures, and to identify appropriate indicators and measures for each workstream or workstream group.
17. All intermediate outcome measures identified at stage (3) should inform the proxy outcomes identified in Table 1 above. Any misalignment would provide a warning flag for evaluation of the workstream to determine whether it is doing the right activities, or what activity is ongoing that is taking it of course towards the proxy outcomes measures. 18. Ultimately the workstreams should inform the vision of where the Commission would like
to be in 3 years (4).
19. To demonstrate this, an example, using the Anti Money Laundering workstream is set out below. Figure 2 demonstrates how outputs (2) link to intermediate proxy outcomes
(3) with associated measures, which in turn feed into proxy outcome measures/levers identified (4) and inform progress towards the vision (5). AML outputs focus on devising internal controls for easy identification of AML issues, whether at the point of licensing or through compliance activity. It focuses on use of information to inform decisions and assurance to the Commission that AML is seen as the responsibility of the industry. Results should be realised through the level of AML cases and increased transparency within the industry on their approaches to AML. Collaboration /engagement and reputation/authority are key tenets to delivering this workstream, ultimately leading to increased assurance that crime is not being associated with gambling.
The diagram shows how one workstream delivers into identified outcomes (or levers). Once this is repeated for all workstreams, we will start to develop a basket of measures that can be used to inform the proxy outcomes measures (4). When a report on
performance is delivered to Board, for example, the industry taking responsibility, it is feasible that the Commission would be in a position to identify all the workstreams that contribute to this outcome, all workstreams have measures which identify how outputs are contributing to outcomes, and at Board level, an aggregation of measures painting a picture of progress made. This will provide a wide overview of the Commission’s activity towards achieving specific outcomes as opposed to taking an approach whereby we focus on individual workstreams and issues in isolation of other activity.
Figure 1: Framework for outcome performance measures Vision (4) Proxy outcome measures (3 outcomes identified) (3) Intermediate outcomes (workstream outcomes identified in
business plan report)
(2) Outputs
(Workstream deliverables) – management information
Next steps
20. As discussed earlier, the process of identifying measures against indicators will take a top down and bottom up approach. This will engage colleagues in the development of corporate measures, and encourage ownership for reporting against each indicator. 21. Colleagues will be engaged through programme leads, who will take responsibility for
demonstrating how their programme (and associated workstreams), contribute to the 3 outcomes identified. They will also be expected to identify measures that contribute towards indicators. It is assumed that programme leads will work closely with workstream leads through this process.
22. An internal brokering process will take place, which will involve programme leads, workstream leads, senior management team to refine measures identified, with the outcome of a basket of measures which feed into the wider outcome indicators. Walter Merrick and Robin Dahlberg will be engaged throughout this process, specifically to seek a steer and guidance.
23. It is assumed that this will be quite a lengthy process requiring several iterations before a draft is available for Board consideration.
Recommendations
24. The Board is asked to note the outline draft framework for outcome performance measures and provide a steer
25. A further detailed framework, with draft measures and a performance reporting framework will be presented at the Board meeting in September.
