EASTERN REGION MAKING A DIFFERENCE TOOLKIT LSCB PERFORMANCE MANAGEMENT HANDBOOK

(1)

1 Eastern Region LSCB Performance Management Handbook – March 2013

EASTERN REGION

‘MAKING A DIFFERENCE’ TOOLKIT

LSCB PERFORMANCE MANAGEMENT HANDBOOK

Introduction

Working Together to Safeguard Children, in effect from 15th April 2013, provides a guide to inter-agency working to safeguarding and promote the welfare of children under Sections 11 (4) and Section 16 of the Children Act 2004. The guidance reinforces the duty of Local Safeguarding Children Boards (LSCBs) to monitor the effectiveness of local services and legislative requirements. Put simply – “safeguarding is everyone’s responsibility”1.

Chapter 3 of Working Together specifically outlines the responsibility for all LSCB members and the performance management function for LSCBs to “use data and, as a minimum, should:

 Assess the effectiveness of the help being provided to children and families, including early help;

 Assess whether LSCB partners are fulfilling their statutory obligations set out in chapter 2 (of the guidance);

 Quality assure practice, including through joint audits of case files involving practitioners and identifying lessons to be learned’ and

 Monitor and evaluate the effectiveness of training, including multi-agency training, to safeguard and promote the welfare of children.”2

This performance management handbook for LSCBs aims to help LSCBs and the constituent members to fulfil this function. It is part of a larger regional toolkit to assist children’s services performance management in the region as part of sector led improvement and has been developed using expertise and examples of good practice both from the region and nationally. However, it does recognise that different LSCBs and their component agencies need differing levels of help and support at different times, and the active engagement of all partners is essential to ensure success.

1_{Paragraph 1, page 7, Working Together (2013), DfE.} 2

(2)

Boards are all operating in very different local area service contexts, provision such as early help and structures. The effectiveness of Boards also varies, and not all LSCBs are at the same stage in their development of performance management.

Key findings and overarching principles from the work in the region in developing the toolkit are:

 A ‘one size fits all’ model is not suitable;

 Boards need to see themselves as undertaking a cycle of performance management rather than looking at performance information.

 Boards should be fully informed of the state of safeguarding in their locality with an aspiration that there should be ‘no surprises’ arising from inspections;

 There is an opportunity for continuous improvement, both within the cycle of each Board’s activity and by sharing good practice between Boards;

 The views of children and young people, families and professionals together with stronger qualitative information needs to be more strongly represented than it is at present;

 There is a need to understand for each Board ‘what good looks like’.

All Boards, and especially those requiring some improvement, may find it helpful to reflect on the guidance, challenge questions and tools that this handbook and associated documents provides to define or refine their own performance management framework. The shared learning journey and ownership by the Board and it’s member agencies is as important as the resulting framework and its implementation.

There are five components to the regional toolkit, in addition to a programme of development activities planned between March and June 2013:

 A Performance Diagnostic Tool, aimed at organisations to self-assess how effective their performance management is;

 A series of guidance and help sheets, providing general guidance, “how to…”, challenge questions and signposts to additional support in this area, and of which this LSCB performance management handbook is a component;

 Tools and templates which can either be used as they are or adapted to suit the LSCB. This includes a model LSCB outcomes framework;

 Examples from other LSCBs, which it is envisaged will be added to by LSCBs over time;



Reference documents, such as QA frameworks and guidance.

(3)

Handbook Contents

1. The starting point: vision, purpose, mission

2. Governance and culture

3. Performance management frameworks: how performance is managed

4. Outcomes and data: what intelligence we need

5. Quality Assurance and Audit

6. Reporting performance: how are we doing?

7. What does good look like?

(4)

This LSCB Performance Management Handbook is supported by the following additional information and tools.

Help Sheets and Guidance

Reference Documents

LSCB focus:

 LSCB Performance Management Handbook Generic regional help sheets:

 Performance management strategy and culture  Target Setting

 Early Help – challenge questions  Measuring Impact

 Data quality

 Quality Assurance and Audit

 Access to data and information sharing  Analysing and presenting data

 Undertaking evaluation to measure effectiveness  Glossary

 Types of data and intelligence

Good practice by Local Safeguarding Children Boards Ofsted, September 2011

Working together to safeguard children DSCF [as was], March 2010

Ofsted - Framework for the inspection of local authority arrangements for the protection of children Ofsted, April 2012

Developing a strategic quality assurance framework to safeguard children

LGID, January 2011

What are the key questions for audit of child protection systems and decision-making? C4EO, November 2009

Canary in the cage? Lead indicators and their potential use by LSCBs

Childhood Wellbeing Research Centre, 2011 Children’s safeguarding performance information framework

DfE, October 2012

The LSCB Challenge and Improvement Tool

DCSF, 2008

Templates

Examples

 Eastern region LSCB Model Outcomes Framework  A variety of performance report templates and

scorecards

 Performance action plan template  Sample information sharing protocol

 Thurrock Performance Framework  Southend Performance Framework  Hertfordshire Performance Framework

 Cambridgeshire LSCB QA group Multi-Agency Review template

 Peterborough Young Peoples Safeguarding Advisory Group Terms of Reference

(5)

1. The Starting Point: Vision, Purpose, Mission

Strategic planning answers the vision question, “Are we doing the right things?” through formulating objectives and developing specific strategies to achieve those objectives. It is future-oriented and focuses on describing how an organisation, such as an LSCB, will adapt to an ever changing environment of demand, legislative and financial requirements together with the needs of children and their families.

Performance management answers the related question, “Are we doing things right?” and also “Are we making a difference” to evidence that:

 we have kept children safe;

 we have improved outcomes for children and their families;

 we have done so in a cost effective, and consistent manner.

Performance management therefore represents a systematic, intelligence-oriented approach to managing and improving people, resources, processes and activities to achieve objectives, within a learning and improvement culture where all agencies and workforce within the LSCB remit can align their own priorities and actions to accomplish the LSCB’s objectives. This can only succeed if there is access to robust, timely data3.

For LSCBs, the annual report and business planning are essential components of effective performance management, setting the vision, purpose and priorities through self-assessment and outlining the strategies to achieve these (figure below).

All measures of performance and quality which are then determined to measure these, in addition to core functions, should then refer back to the vision and priorities set through business planning, and it should be clear for each how it answers the question 'how does this help us to know that we are doing what we need to do?'.

3_{There are help sheets within the Eastern Region toolkit which describe different types of data, data quality,}

(6)

Without this central golden thread it is all too easy to become distracted by other possible performance or quality issues, or to arrive at a point where an overwhelming amount of information is collected and reported either because it always has been, or simply because it is there, as is often the case with a disproportionate reporting of children’s social care data to other agency data.

Strong LSCBs will ensure they are assured of the performance management arrangements within their member organisations and other services within the local area, and there is systematic and effective scrutiny of a range of intelligence without it being burdensome. This may include some data which has already been dealt with within member organisations, and/or the ability to undertake a ‘deep dive’ to gain that assurance.

Structure/ Culture Governance, sub groups, resources, business management Annual Report and Business Planning Performance Monitoring Duty to safeguard (S11) Statutory Responsib-ilities Self Assess-ment

Challenge Questions

 How clearly is your vision articulated? Who is involved, and where and how is this done?  Is the annual report based on a range of evidence across all partners?

 Does the annual report reflect how effective the Board has been and it’s impact in maintaining and raising standards, as well quality of safeguarding?

 Can all elements of performance reports to your Board (including sub-groups) be clearly linked to the Board's objectives and priorities as laid out in the Annual Report?

Agency Review Action Plans Scrutiny and Review

Quantitative and qualitative data, case audits, views of CYPP and professionals, complaints, evaluations

Policies and Procedures Training SCRs CDOP etc Within partner agencies

As a Board Peer challenge (other LSCBS)

(7)

2. Governance and Culture

Having the right structures and governance in place is no less important in performance management than in any other aspect of the operation of an effective Board. Indeed good practice suggests that this structure should include a sub group with a specific focus on performance, quality assurance and audit which receives and analyses the full range of intelligence and ensures partnership information sharing at both child level and performance data level is robust and acted upon. Whilst the titles of these groups where they are in existence vary, it is recognised they should be a ‘learning and improvement group’.

The structure should support the Board's ability to challenge itself and its partner agencies. This in itself may prove challenging for the officers involved who, whilst likely to be involved in challenging practice within their own agencies, may have also be used to taking a more defensive stance in the face of external challenge. Relevant staff need to understand their roles within this process and be given the freedom to exercise them appropriately.

The right culture is vital. The strongest LSCB have a learning culture rather than a blame culture, where individual agencies are able to be an active partner in the LSCB and to understand and articulate the intended and actual impact of practice across partners, contributing to the story behind the data, emerging hypotheses and any corrective action planning. Individual agencies in the strongest LSCBs are able to manage their own performance, manage risks and report on an exception basis to the LSCB within agreed governance so that there is a robust accountability and escalation of performance issues to the LSCB.

An effective LSCB will also work strategically with the Health and Wellbeing Board together with other partnerships in the local area in respect of the shared agenda for the well-being of children and young people and their families. This may be evidenced by joint projects, shared intelligence and collaborative improvement planning where appropriate.

It is important to recognise the added value of the LSCB as challenge and support to each other as a body, but also the opportunities within a multi-agency framework where the Board is able to aggregate information to identify needs and outcomes of populations and cohorts of children and families across services (i.e. a child centred, outcomes based accountability approach) but also pan-agency workforce and resource data (for example, is there a high level of vacancies across all professions or in one agency, and if so, what corrective action can be taken collectively?)

Governance and performance management of the LSCB will also reflect compliance with current legislation and statutory guidance, and be assured that any changes are undertaken with constituent agencies effectively. This toolkit provides links to some of the current guidance and legislation for LSCBs at the time of writing and in addition, further support can be found in “Good Governance for Boards” by OPM4

4

(8)

3. Performance Management Strategies: How Performance is Managed

To be effective, LSCBs will have a strategic and systematic approach to performance through a framework which describes how they will manage performance, and how it will be an integral part of their learning and development strategy. In addition, Boards will need to consider ‘what intelligence we need’ (see next section). Whilst the LSCBs approach to how they will manage performance may not change overtly, what intelligence they need could differ depending on current national and local priorities.

Both should strongly relate to the annual report and business planning as previously described; and the approach should include clear processes; responsibilities of individuals, sub-groups and partner agencies; a timeframe for a cycle of audit/scrutiny and how improvement activities will be undertaken. It should evidence that the LSCB is a learning organisation and should be jointly agreed and actively followed by all partners.

This section shares some commonly used or effective types of performance management strategies and methods, including some approaches such as Outcomes Based Accountability (OBA) which are covered more fully in other parts of the toolkit.

Challenge Questions for the Board

 Could your organisational structure support performance management more effectively?  Does the structure include a multi-agency group of staff who deal with performance

information in their own agencies and who are skilled in presenting, analysing and interpreting data?

 Does the main Board and its sub groups within your structure contain the right people, dealing with the right business and receiving the right data appropriate to their focus, which they can understand and act upon?

 Are performance management, quality assurance and audit consolidated into one function in your LSCB, and do the results feed into workforce development?

 Are there clearly understood exception reporting thresholds between individual agencies and the Board, so that there is transparency and areas of potential or actual risk or declining performance are not missed?

 Is there a learning culture within your LSCB so that the main Board and sub-groups are able to support and challenge effectively?

 Do members of the LSCB, especially those who may not be resident in the local area, fully understand the local context and the communities which they serve?

 Challenge questions that LSCB members may wish to reflect on to understand whether the LSCB functions as a single accountable body with all partners actively engaged, are:

o Is there a direct link between my involvement in the LSCB and the protection of children and young people?

o What contribution will I make?

(9)

9 Eastern Region LSCB Performance Management Handbook – March 2013 3.1 Outcomes Based Accountability

Outcomes Based Accountability is a performance methodology originally developed by Mark Friedman and popular amongst public bodies5. It allows us to articulate what we want for children and young people – to start with the end (what is our goal) and work backwards to how are we going to achieve it. The framework clearly distinguishes between what we do, and what effects that produces, and between the quality and quantity of each. A key component of the framework is the focus on the question "Is anyone better off as a result?". The components of this methodology, and the questions generated (see figure below) help LSCBs to measure their impact and provides ‘report cards’ for ensuring the right people are looking at the right data and having meaningful discussions to generate improvement activities.

QUANTITY QUALITY

EFF

ORT _{How much did we do?} _{How well did we do it?}

EFF

ECT Is Anyone better off?

Quantity Percentage

CH

AN

GE _{Action planning:}

What do we do to improve and when will we know we have?

3.2 Examples from LSCBs

A number of examples of LSCB performance frameworks are provided within the toolkit, with some of those from the region described below. However, if considered as an element of the LSCB performance framework, the diagrams in themselves will require accompanying narrative to provide the detail for the Board.

Southend’s LSCB performance framework has been provided as an example of effective practice. ensures that the LSCB’s single agency performance data and multi-agency audit programme will support the collection of the quantitative and qualitative data required as well as identifying and managing risk. It consists of eight elements:

1. Serious Case Reviews – statutory process currently under review by government 2. Alternative Case Reviews – local process developed and reviewed by Case Review Panel 3. Child Death Reviews – statutory process currently under review by Strategic CDR Overview

Panel

4. Section 11 Audits – regionally developed process to fulfil statutory requirement

5. Single agency performance data – locally developed process to be reviewed by Monitoring Sub Group

6. Multi agency audit programme - locally developed process to be reviewed by Monitoring Sub Group

7. Performance information to the Board – which is regularly reviewed and revised.

5

(10)

8. Themed Reports – local process developed by the LSCB Executive and not currently under review

Thurrock ‘wheel’ is an example which represents the approach taken by the Thurrock and Essex LSCBs. The approach is based on six questions, one in each of the inner circle sections of the wheel. The next circle out describes the methods used for assessing performance / progress against these questions, and at the outer edge are positioned indicators which will be monitored as part of this process. Procedurally, each Board meeting addresses one of the questions in turn. This model has some elements which are similar to the OBA approach. A template for the model is provided in the toolkit. Is the children’s workforce fit for purpose? Is safeguarding really everyone’s business?

Do the right children have protection plans and are they being fully implemented in a timely way?

Are we sure that no two children will die as a result of the same system failure/s?

 Section 11 audits

 Allegations data

 Implementation of safer recruitment standards

 Workload and retention thematic reports

 Safeguarding training data

Are we satisfied with the quality of care for any child not living with its parent, a close relative or someone else with parental responsibility?

 SCR data – number and themes of case

 SCR recommendation implementation  SCR recommendation analysis –

repeat recommendations  Child death review data  Child death

review data

Performance monitoring of 6

key questions

Are we doing all we can to reduce the risk of avoidable child deaths?  CP statistics  Improvement activity – CP conferences / SCF auditing  Multi-agency audits  Reports of LSSGs

 Children subject to plan  Categories of plan  Length of time on plan  CP plans for a second time  ICPC leading to plan

 Percentage of agencies good / adequate / inadequate – section 11 audits  Referrals by source organisation  Referrals by source organisation that go to IA  Attendance of professional at ICPC and provision of reports  Completed CAFs  Section 11 compliance results  Referral statistics  CAF / Integrated working data  Child and parent/carer survey data  Annual PF report

 Thematic reports re prisons / YOIs / secure settings

 Thematic report re missing / runaway

 Thematic report re migrant / trafficked

 LAC group report

 No. child deaths  No of preventable deaths  Cause of death  Area of death

 Percentage staff with CRB checks

 Designated staff vacancy levels

 Percentage of staff trained to required identified group, per group  No. of notified private

fostering arrangements  Looked after children by

placement type  Children notified as missing

 Percentage recommendations implemented and implemented to timescale

 Percentage of repeat recommendations / recommendations with a repeated theme

 Number of deaths indicating service provision as a notable factor

(11)

11 Eastern Region LSCB Performance Management Handbook – March 2013 Hertfordshire Model

There is also some similarity too with the Hertfordshire model where six high level outcomes are identified, with under each a set of strategic measures, context measures, and process measures. This is supported by the performance framework illustrated below.

(12)

12 Eastern Region LSCB Performance Management Handbook – March 2013 3.3 Levels of reporting

Any performance framework should be clear about what levels of information are required by the Board, its sub groups and within individual agencies. A common feature of Boards who do not use data effectively is that they are ‘data rich, intelligence poor’, where a large quantity of data is presented directly to the main/top tier Board with little ‘story behind the data’ and insufficient time for discussion about what it means and what corrective action planning is required.

A phased approach within a performance management framework as in the figure below would ensure optimum effectiveness of the Board as a whole, by allocating responsibility within the structure’s sub groups and individual agencies for a range of data and intelligence.

The place of information within this hierarchy is not static. For instance Boards require the agility to respond to emerging themes and new priorities, which may necessitate the inclusion of new performance information from time to time. However when this happens, setting time limits for the change should be considered where possible.

LEVEL 3: MULTI-AGENCY REPORT

OR SCORECARD High level summary statistics and performance indicators or measures required by the Board for regular monitoring. Exception reporting.

LEVEL 2: LSCB SUB GROUPS / THEMED REPORTS More detailed information which may be used by sub groups

for specific purposes, or which may be required to support themed audits. May be reported to the Board by exception

when necessary.

LEVEL 1: AGENCY

Detailed information which is reported and measured within partner agencies. Whilst not usually required by the Board such information may be requested

(13)

4. Outcomes Data: What intelligence we need

A key decision for LSCBs is what performance measures they require to ensure a balanced partnership view that covers statutory responsibilities and reflects all partners, as well as local priorities. There are currently a number of outcomes frameworks which provide LSCBs and partner agencies with performance measures, but as these tend to change periodically and in some cases are likely to require a response at agency level only, the LSCB should be ready to consider and act accordingly to relevant new frameworks and performance measures whilst maintaining a focus on what is important to them and their local area.

4.1 National outcomes frameworks

The list below is not an exhaustive range of outcomes frameworks currently in existence. In addition to these, there is a vast range of data available through statutory data collections and the full range of partner agencies. More detail about the latter is provided in the ‘accessing data’ help sheet as part of the regional toolkit.

 DfE Children’s Safeguarding Performance Information Framework: DfE issued a significant document for LSCBs and those involved in safeguarding children in October 2012: Children’s Safeguarding Performance Information Framework6. This is a list of key information, describing the key nationally collected data (based on the Munro dataset) that can help those involved in child protection at both the local and national levels understand the health of the child protection system. It also describes recommended questions that can be asked of children, families, professionals and providers at a local level. The framework is broken down into five themes:

 Outcomes for children and young people and their families

 Child protection activity (including early help)

 The quality and timeliness of decision making

 The quality of child protection plans

 Workforce

6

http://www.education.gov.uk/b00209694/perf-info

Challenge Questions

 Does your Board have a written and agreed strategy how it will manage performance which includes clear accountabilities; processes; timescales and encompasses all elements of performance management and quality assurance?

 Does it reflect the child’s journey across services and involvement of intelligence from all agencies?

(14)

Accompanying this, DfE have provided advice about locally held information7 and how local areas collect information and use the recommended local questions.

 NHS Outcomes Framework 2013-20148, published November 2012 by Department of Health contains measures to help the health and care system to focus on measuring outcomes. It highlights the main indicator changes across each of the five domains from previous versions and is accompanied by a Technical Appendix which provides detailed information about each of the indicators.

 A public health outcomes framework for England, 2013-20169, published January 2012 by Department of Health, includes public health indicators grouped under four domains.

 Children’s and young people’s health outcomes forum10, report published in July 2012 provides recommendations on how health of children and young people can be improved and proposes a number of new outcomes measures and the strengthening of existing indicators, and makes specific recommendations for different organisations within the health and care system to ensure the improvements are achieved.

 Social Justice Outcomes Framework11, published by HM Government in October 2012, following publication of Social Justice: Transforming Lives, this outcomes framework highlights priorities, plans to measure progress and seven key indicators covering supporting families, youth crime, and delivering social justice.

4. 2 Eastern Region LSCB Outcomes Framework & Core Dataset

The range of performance measures required by children’s services (e.g. Ofsted inspection, outcomes frameworks above) has been mapped and illustrates the significant number (over 200) that are required to be reported relating to safeguarding, which is obviously unrealistic and undesirable for an LSCB to monitor.

In selecting measures for their own local core data set, Boards should be mindful of:

 The rationale for including a measure - how does it support your work? Is it a performance measure that needs to come to the Board, or that can be monitored in a local agency and not duplicating arrangements already in place within partner agencies;

 Ensuring an acceptable levels of data and information required by the main Board and by its sub-groups; 7 http://www.education.gov.uk/childrenandyoungpeople/safeguardingchildren/protection/b00209694/perf-info/local-info 8 https://www.wp.dh.gov.uk/publications/files/2012/11/121109-NHS-Outcomes-Framework-2013-14.pdf 9 http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_132559.pdf 10 http://www.dh.gov.uk/health/files/2012/07/CYP-report.pdf 11_{http://www.dwp.gov.uk/docs/social-justice-outcomes-framework.pdf}

(15)

 What needs to be part of the monitoring of 'business as usual', and what needs to be reported on a 'by exception' basis;

 What supports the identification of risk;

 What information [be it business as usual or by exception] supports the Board in challenging agencies, and practice within agencies;

 That qualitative information and commentary, results from evaluations, audits, complaints etc. are included as well as statistical measures.

 Does it support knowing “are we doing the right things?” and “are we making a difference?”. To assist LSCBs in the region to identify a core set of key performance measures which have national, generic significance for LSCBs, a model outcomes framework has been developed and this is described more fully in the last section of this handbook.

4.3 Types of evidence and sources of data

The strongest LSCBs will consider a range of evidence to ensure there is robust intelligence available which is triangulated, such as:

 Data - information, especially facts or numbers, about the children and young people in the local area; receiving services; information about the activities and outputs; resource (finance and workforce) information; outcome information. Data can be qualitative or quantitative, and include results of quality assurance activities and audits.

 Professional expertise and research - What you know about the people in your area and professional judgement, knowledge of other professionals and colleagues, and evidence from research and evaluations.

 Voice of the Customer - What services users tell you about how you treated them, what they learned, have you made a difference, what could have been better. This should also include children and families who are NOT receiving services (to identify whether needs of the population are being met and there are no gaps in achievement of outcomes), and communities.

Challenge Questions

 Are members of the Board aware of the relevant outcomes frameworks in their agency and share these with the Board?

 Are you aware of all the relevant current outcomes frameworks which impact safeguarding for children and young people? Do you have systems in place for timely access to, discussion about and appropriate action on new frameworks and performance requirements which may arise in the future?

 Does the Board have a proportionate and manageable approach to selecting outcomes and measures for these which reflect the criteria in 4.2 above?

(16)

Some of this information will be readily available from data systems, case records, etc, but some will require additional data collection such as surveys and information gathering. Quantitative data (how much did we do?) is usually easier to obtain from data systems and the type of routine monitoring agencies will already be using either internally or for external reporting. Qualitative data (how well did we do it, and is anyone better off as a result?) is more likely to require a mixture of system generated data and some form of dialogue with staff and service users, the voices of both groups being a vital part of the evidence base.

LSCBs and constituent agencies will find benefits to identifying and utilising intelligence that is already available rather then re-inventing new evidence. For example, there are likely to be evaluations, feedback from complaints, customer feedback and regular data monitoring that can be suited into the LSCBs performance management.

Additional methods and tools of measuring impact and outcomes are contained within the full performance toolkit include distance travelled tools, social return on investment models and evaluation methodologies.

4.4 Related initiatives – same children and families, different tag?

An alternative way to identify whether we have made a difference to children and their families is to take a cross-agency population accountability view – i.e. look at all the services and inputs around the child and family across services, and measure the impact it has collectively, and as component parts. This may be especially valuable if the child/family is receiving a number of different interventions, all asking the child and family the same/similar questions and defining similar measures. For example the same child/family may be a focus for school; health; youth offending; troubled families initiative; child poverty; housing; etc. This area based approach to understanding performance of the ‘team around the child’ involves agreement and synergy between all services, including schools, and is well supported by the CAF model.

Some information also has the potential to include new views of existing data which can be created by bringing together data from different agencies, for example police data on domestic abuse reports combined with data on child protection plans and overlaid using mapping technologies. Such examples demonstrate ways in which LSCBs can add value to existing information by acting as both the commissioner and conduit for the joint use of information and data by partner agencies. An information officer sub group is a useful route for exploring these possibilities.

Challenge Questions

 Does your current dataset include information which is no longer of relevance or is reported because it always has been (e.g. National Indicators)? Are you clear that your selected indicators reflect your priorities and key measures from outcomes frameworks?

 Do you know what existing evidence is available (such as evaluations) and are you ableto access it? Will existing measures tell you what you need to know, or is it necessary to develop something more appropriate?

(17)

5. Quality Assurance and Audit

Quality Assurance refers to all activity that contributes to service provision and improvement through satisfying the organisation that certain standards are being met. It should be an integral part of performance management, bringing in a qualitative approach to set and monitor standards to ensure that all children and families receive a consistent and appropriate standard. Data, audit, evaluation, supervision and customer feedback all assist in providing the evidence.

Quality Assurance will generally consist of: • Practice standards

• Workforce processes (Supervision, appraisal, management oversight and authorisations) • Case file audit programme including themed audits.

• Schedule of planned or unplanned peer or other audits and reviews, including serious case reviews and management reviews.

• Complaints monitoring

Results of audits should feed into performance reporting and workforce development, but also and a programme of re-audits to measure the change (measuring impact – have we improved from first audit to re-audit?)

Strong LSCBs also use their performance monitoring, where performance is declining or coasting, to help shape the audit programme to identify more detailed information to assist service improvement.

Further guidance for LSCBs is widely available and therefore not replicated here (see contents for list of guidance).

 Where is it helpful to be able to compare data [therefore needing a common definition]?  Have you included a range of qualitative data and ‘not just numbers’?

 Are you able to identify and undertake cross-agency data analysis where appropriate to obtain a better view of the child’s journey?

Challenge Questions

 Are your Q&A and audit processes embedded within your performance reporting?

 Are your performance management, quality assurance and audit aligned to avoid duplication and ensure robust improvement mechanisms?

 Who undertakes Audits and are they skilled and supported in doing so?

 Do you feel you have the right quantity of audit activity to commonly agreed standards across agencies?

 Do you have a robust process for picking up lessons learnt and go back and audit or re-measure to assess whether improvement has been made?

(18)

6. Reporting Performance: How are we doing?

How performance is reported is critical to ensure the audience is able to understand what it is telling them and thereby produce robust hypotheses and conclusions as to whether corrective action is required to improve performance. There are help sheets and tools available as part of the Eastern Region toolkit which provide further detail and examples of reports, but some key features of successful performance reports are:

The following give reports their integrity:

 Data used is credible, comprehensive, and of high quality (where there are known data quality issues these should be identified and addressed).

 Appropriately skilled data analysts should identify patterns and trends in the data. The following give reports meaning:

 Reports should be mindful of their audience, and the priorities this audience has responsibility for.

 Reports should tell the story born of the analysis in a way that responds to the audience's interests.

 There should be clear headings and description of what the data is and what period it covers.

 Reports should include ‘the story behind the data’ in words and context: remember that data is describing children or services provided for them and should be written in that form.

The following ensures that the form and function of the report are fit for purpose:

 Any visualisations should be of the appropriate type for the data / information represented.

 Good reports avoid cluttered or visually confusing presentation and the temptation to present every available piece of information all at once.

Well presented performance data will usually include:

 A table of the data alongside a visualisation (such as a chart) where appropriate - most audiences will include some who prefer tables of raw data alongside those who can more easily assimilate graphically presented information.

 An indication of what good looks like and whether this is absolute (a fixed point such as '100%'), relative (such as 'an increase over the previous year') or within an expected range such as 'between 20 and 30').

 Comparisons where appropriate, e.g. with national data, with comparators, with available definitions of 'good', or with targets where these are set.

 An indication of direction of travel over time.

It is often helpful to split performance reports into summary (dashboard or scorecard) and detailed sections, so that those who need an overview can see the important headlines in the summary, whilst the detail is available for those who need it or who have a particular interest in specific areas. A common problem with performance reports is that they become unwieldy, especially as items are added and none removed. It may be wise to set a limit, such as the number of performance

(19)

measures per outcome, number of statistics per performance measure, lines on a graph, etc. Millers Law12, also known as the rule of seven, suggests that the maximum number of pieces of information that a mind can process effectively at one time is seven.

Some elements of performance reports will not fit into a data style reporting format. For instance the results of a themed audit will probably require a more narrative presentation. However, specific findings from such an audit might themselves form part of the professional commentary against a specific measure, e.g. findings from a themed audit of domestic abuse cases might yield information pertinent to the previous example around staff training. So whilst the narrative section should report the main findings, don't forget to cross reference where appropriate.

7. What does good look like?

What good looks like may differ between LSCBs and agencies depending on their current performance in key areas and their priorities. This section looks at three aspects:

a) What does good performance management and QA look like? b) What does a good LSCB look like to Inspectors?

c) What does good safeguarding provision look like?

7.1 Good performance management and QA

The excerpt below from the regional performance management and QA diagnostic illustrates how an effective performance management system underpins the way an organisation is run. The full diagnostic tool provides organisations with the means to self-assess how effective they are against key criteria.

12_{In 1956 George A. Miller of Princeton University's Department of Psychology published a paper in which he}

suggested that the number of objects an average human can hold in working memory is 7 (± 2)

Challenge Questions

 Do your current performance reports effectively support the work of the Board?  Do they contain the right level and elements of information as described above?

 Is there benchmark data available and are you able to identify how you are performing?  Are reports clear and well presented?

 Is it clear what the data is telling you, and easy to understand ‘the story behind the data’.  Is there evidence in either meeting minutes or in the reports what action, if any, is being

(20)

1. Performance management and quality assurance are led as the basis on which the organisation is run and improvement is made and evidenced.

2. All LSCB members understand the performance management system, talk about it easily and know how it fits into their work.

3. It is recognised that a range of intelligence, including are required for performance and improvement and members are able to review, analyse and challenge this.

4. Members receive timely performance information and use it to inform their decision making both of the Board and fed back into their own agency.

5. Risk assessment is used to inform business planning and risks are managed as an integral element of performance management.

7.2 What does a good LSCB look like to Inspectors?

This section provides a synopsis of Ofsteds current inspection framework on which LSCBs will be judged. The new multi-agency inspection framework which will commence in 2013 is still undergoing refinement as at the time of writing this handbook.

Ofsted's Current Inspection Framework

Leadership and Governance "This encompasses leadership and governance within the local authority, through both elected members and officers, and at both corporate and departmental levels, and through the Local Safeguarding Children Board (LSCB). It will evaluate how effectively the local authority leads partnership working in relation to early help and child protection, and the effectiveness and capacity of leadership in achieving and sustaining improvement."

The framework requires that:

 there are clear accountabilities and responsibilities between the LSCB, the Director of Children’s Services, the Chief Executive, the Lead Member for Children’s Services, and all other partners

 the LSCB meets its statutory duties and is effective

 performance management and evaluation is effective

Grade Descriptors

Outstanding

 The leadership and governance judgement is likely to be outstanding if all the requirements for a good judgement are met, in addition to the following.

 The effectiveness of the local authority’s leadership is recognised by partners as driving improvement across the whole system.

 There is clear evidence of a shared priority to delivering effective help and protection to all children at risk of harm, given by all agencies and services, and this is translated into action.

(21)

 There is a shared and effective approach to performance management and improvement across all agencies.

 All agencies and services work together as a single system to drive improvement in the effective protection of and help for children and young people and in improving outcomes for them.

Good

 The LSCB provides effective challenge to all partner agencies, which drives improvement.

Adequate

 There are clear accountabilities and responsibilities between the LSCB, the Director of Children’s Services, the Chief Executive and the Lead Member for Children’s Services, to ensure that sufficient and focused attention is given to children and young people who are suffering, or likely to suffer, harm from abuse and neglect.

 The LSCB has been effective in improving the quality of child protection practice across the system, and all key partners are committed to its work.

 The LSCB ensures that multi-agency training is available and effective in improving the protection of children and young people.

 Performance management and evaluation is established within the local authority and in partner agencies, and leads to improvements in keeping children and young people safe from harm, and in outcomes.

Inadequate

 The leadership and governance judgement is likely to be inadequate if it fails to meet the requirements for an adequate judgement, and deficits in leadership and governance contribute to failure to provide effective help and protection for children.

New Inspection Framework [from April 2013]

Ofsted presentation to the National Children and Adults Conference in October 2012 stressed the role of LSCBs in challenging practice. Ofsted state that:

 inspection could be used as a lever to support more co-operation between partners, but acknowledged that the weighting of each agency’s contribution would be complicated and there were no solutions as yet. Inspecting the LSCB would allow partnership working to be central to findings, rather than each partner individually.

 “ the expectations on the LSCB would be higher, in that it would no longer be sufficient to have good high level attendance at LSCB meetings, rather LSCBs would need to demonstrate how they scrutinise practice and how far partners understand the weaknesses in the system."

(22)

The Ofsted Report 'Good Practice by LSCBs' (September 2011) acknowledges that: "The link between Board activity and specific outcomes for children is not always an easy one to make as there are multiple factors involved. However, some Boards are using measures which are beginning to demonstrate that they are having an impact. Sometimes the link is not a direct one, and the improvements in outcomes are implied rather than measurable. This is due, in part, to LSCBs being unable to track children over a long period (unlike research) and because, to a large extent, many of the outcomes are preventable ones, in other words things that have not happened; for example an injury to a child or a child death, criminal activity, teenage pregnancy or substance misuse. While Boards are beginning to measure whether there has been a reduction in these activities, these are inevitably proxy measures and it is difficult to provide a direct link between the reduction and the activities of the Board."

7.3 What does good safeguarding provision look like?

Discussion with the region's chairs and business manager has led to the development of a further model drawing on some of the elements above to gain a more dynamic view of where and how change needs to come about – through practice, management, inter-agency working and partnership or the Board itself. The model asks one fundamental question from four different points of view:

What does good look like...

 ...to the child?

 ...to the team around the child?

 ...to the agency around the team?

 ...to the Board around the agencies?

Under each question a set of outcomes has been agreed, and against these outcomes the question 'how will we know' is framed by examples of different types of evidence - data, audit, voice, and other evidence so that robust triangulation can be made to answer the question ‘what difference are we making?’.

The strongest LSCBs will determine what good looks like for them and the toolkit’s target setting guidance may assist.

Challenge Questions

 As a minimum, do you meet Ofsteds adequate judgements and do you have plans in place to generate further improvement?

 Does the Board have processes in place to access, discuss and formulate required actions to meet the criteria of the new inspection framework?

 How does your Board demonstrably challenge practice and how do performance reports support this?

 Do your Board members understand what good looks like for your particular local area and individual performance measures?

(23)

1. Systems that work

2. Intelligence is available and accessed appropriately

3. All staff and managers have skills and capacity to understand data

4.Robust Self Assessment, Annual Report and Improvement Planning is

undertaken

8. Using the handbook and toolkit

Keeping the handbook and toolkit current to reflect national changes and support peer development will be necessary and under the regional sector led improvement programme, but LSCBs are encouraged to share examples and methods as well as data and trends where possible to promote peer support and learning.

LSCBs are encouraged to use the handbook and tools and share with partners to ask some of the challenge questions posed within. The diagram below illustrates how the toolkit can assist in assessing whether performance management is as effective as it can be in four inter-connected areas, which will ensure any organisation knows whether they are making a difference.

Date: April 2013

Owner: Eastern Region Children’s Improvement Programme Diagnostic Tool

Handbook and Challenge Questions

Examples Tools and Templates

Reference Documents  Are performance discussions and decisions included in minutes of meetings or other

documents to provide clear evidence of discussion, action and effectiveness?

 Do you have a record of continuous improvement, and/or of tackling performance issues effectively?

 Is performance management and quality assurance in the LSCB effective?

Do you know whether your Performance Management is effective? 1. Undertake diagnostic 2. Use relevant parts of the

handbook and guidance 3. Look at examples from

others

4. Adopt or adapt any of the tools or templates 5. What other support do

you need, or what do you do well that you can share with others.

We know whether

we are making a difference