Case study:
Case study:
The evolution of the Work Programme
Statistics
Statistics
Adam Pearce
January 2014
Adam Pearce – January 2014
The Work Programme is quite complex…
Attachment
Referral
Outcome Payments
Completion
Assigned
Payment Groups 3-9 Job Outcome payment
paid following 13 weeks in employment. Early completion Assigned to one of 9 Payment Groups Attachment Feepaid to the Provider
Sustainment payments paid following each further period of four weeks in employment up to a maximum of 20 (except Payment Groups 7 and 8 where the maximum is 26 and 13 respectively).
Early completion.
Final eligible outcome payment Complete 104 weeks Randomly allocated to a Provider in their “Black box” provider support begins
Job Outcome payment paid following 26 weeks in employment. S t i t t id Payment Groups 1-2 Outcome payments continue to be paid beyond 104 weeks allotted time. Eligibility ceases following a break
area. Sustainment payments paid following each further period
of four weeks in employment
up to a maximum of 13.
g in employment
…the key measures are relatively complicated…
Departmental Business Plan Transparency indicator
The proportion of those referred to the Work Programme each month for whom providers were paid a Job Outcome payment
Annual Minimum Performance Levels (MPLs)
The number of job outcomes providers must achieve in a year as a proportion of the number of referrals received in that year The proportion of those referred to the Work Programme each month for whom providers were paid a Job Outcome payment within the following 12 months.
The number of job outcomes providers must achieve in a year, as a proportion of the number of referrals received in that year. This is part of the contractual agreement between DWP and providers and is assessed each financial year.
…and the range of stakeholders is challenging …
N i l A di Offi
Minister for Employment
DWP Account Managers DWP Press Office
Researchers
National Audit Office
Number 10
Permanent Secretary for DWP
Charities/Not for profit organisations Employment Support
Organisations y
Work Programme Providers
Public Accounts Committee Devolved administrations
HM Treasury Policy Analysts
Work
Programme
Statistics
Academics ad st at o s PolicyAdvisors Payment Validation Team Cabinet Office
Academics
General Public
Journalists
Work and Pensions Select Committee Employment Services Trade Body (ERSA) Advisors Payment Validation Team
Local Authorities Journalists
First Release November 2012
…so the statistics release has been interesting…
-Limited statistics with a focus on cohorts.
-Detailed background info note.
June 2013
- Included lots more statistics and a LOT ofIncluded lots more statistics and a LOT of commentary
- Moved to quarterly
- Improved timeliness three months on arrears - Aligned to financial years (involved changing
ti l lid ti k) National Audit Office report
December 2012
P bli A t C itt
UKSA monitoring review
- Summarized the views of key users
operational validation work)
September 2013 - Revised layout
Less dense Public Accounts Committee
February 2013
y
- Highlighted scope for further development - range, timeliness, regularity
- public presentation
- information to evaluate provider performance
- Less dense
- Better commentary - More user friendly - Statistics on completers performance
- Concluded that cohort measures were most relevant
Met with NSO Good Practice Team July 2013
P bli lt ti y
Public consultation August 2013
And we have learned some important lessons along the way (part 1)
• Sounds obvious but the title of the release should immediately tell the user what the statistics are and when they are from.
• Our first release of outcomes was helpfully just titled “First Release”Our first release of outcomes was helpfully just titled First Release
• The last release was titled “Work Programme Official Statistics to June 2013”
K k fi di h t ( t if ibl ) i l l t t l h 10 l
• Keep your key findings short (one sentence if possible), simple language, try to only have 10 or less (not everything is key) and they should be on the front page!
• Our June 2013 release’s key finding appeared on page 3 and went on for a page and a half.
Th l t l h d 10 k fi di i th f t d 9 t f th 10
• The last release had 10 key findings appearing on the front page and 9 out of the 10 were one sentence.
M k l h t b d ith th b
• Make sure you are clear over what can be done with the numbers.
• For our first release we didn’t mention the contractual measure associated with the Work
Programme. This was to try and get users to use Cohort analysis as it was a much more accurate
measure of performance. Journalists calculated the MPL anyway and got it wrong!p y y g g
•Later releases all talk about the MPL, stating the reasons why it was created and the reason why it
is a poor measure of performance alongside the reasons why users should use cohort analysis. We p p g y y
And we have learned some important lessons along the way (part 2)
• A picture paints a thousand words.
• Early releases had a large introduction and parts of the key findings/commentary were dedicated to describing the customers journey through the programme.
to describing the customers journey through the programme.
• For subsequent releases this was changed to one flow diagram (that was at the top of slide 2) on page 2 of the release. This was a much neater and simpler way to explain the complexities of the page 2 of the release. This was a much neater and simpler way to explain the complexities of the programme.
•The amount of content and its layout in the release is important too. A packed release that has very littleThe amount of content and its layout in the release is important too. A packed release that has very little white space looks daunting and is likely to put someone off.
• The June 2013 release was approaching 50 pages long with pages upon pages of commentary containing lots and lots of detail
• Later releases are down to 30 pages, with no more than half a page dedicated to commentary, forcing us to be concise.
• Most importantly, you can’t please everyone. The Work Programme release have high ministerial interest and are the public figures for the work private companies have done for the department. Even
ith ll th i t d t ki ' t b d till t iti i d ith
with all the improvements and taking everyone's comments on board, we still get criticised with every release for being both pro and against the Work Programme at the same time!
Further tips
• If you want to use Acronyms then explain them first. Its easy to think that everyone knows
what you mean by JSA, ESA etc
• Rather than percentages try and use X out of 10 etc as it is an easier concept for people to
grasp
• Try to get someone who doesn’t work in your area to go through the last release and see if
they understand it. Its very easy to assume knowledge of the scheme without realising you are doing it.
• Use the National Statisticians good practice team – its what they are there for!!!