As described inConsensus exercise methods, there were four principal sets of discussion questions considered during this meeting:
1. What sort of tool will be most useful to practices (e.g. areas covered, frequency of data entry and feedback)? What unique selling point might make it most attractive for use in the future?
2. How should comprehensiveness and usability be weighed up? What is the right balance between covering core work and more aspirational performance?
3. How can it be ensured that the tool is best suited for use 5–10 years into the future? How can best use within federations be ensured (e.g. practice-level or federation-level data)? How can it be ensured that theFive Year Forward View65and STPs are taken into account?
4. Regarding the content of the tool, which areas should be included/expanded? Are there any other areas that should be covered? How can best use be made of existing data?
A summary of the responses and discussion for each question is given in the following sections.
Question 1
The consensus among participants was that the tool should be considered primarily as a tool to allow practices to measure and improve their own productivity, rather than something for broader performance measurement. It was thought to be very unlikely that standardisation could be applied that would make it directly (and fairly) comparable across different types/sizes of practice.
There was also consensus that the tool should emphasise actionable outcomes, as practices would want to see benefits at little or no cost or extra effort. Feedback should be provided at an appropriately broken-down level to ensure that they can identify the right areas to focus on.
One suggestion was that practices might be able to set their own targets. This might improve a practice’s feel of how in control they are. They could also set their own contingencies.
The process of collecting and entering data must be as straightforward as possible, and take as little time as is feasible. Having the right infrastructure around the tool would be important (e.g. providing automatic extraction of clinical data).
All staff would need to be on board to make it a success, but particularly practice managers and GPs. Some staff (particularly GPs) might need to be convinced about the importance of focusing on some non-clinical or non-core areas.
It is worth noting that the data could, in the future, be used to underpin inspection processes (perhaps under the‘well-led’self-submission for CQC). However, the tool needs to be aligned to the vision for the future of general practice overall; it needs to be part of a coherent, consistent picture.
Concerns were raised about data being used by other bodies; sensitivity is required for practices to use the tool in a meaningful, honest way. However, it could be a very useful tool to help CCGs work with practices (or to compare practices within federations). This points towards a real tension.
It was suggested that in the future it could involve an Indicator Assurance Group, which would have legal responsibility to provide assurance on indicators; this could include involvement from all key national organisations.
Overall, for this question, the key themes that emerged were the need to:
l focus on the measure as a tool for practices to track and improve their own productivity
l enable data collection without much extra burden
l convince practitioners of the value of the tool and enable them to use feedback effectively
l ensure that the tool is situated within the broader, changing context of the NHS, and the health of populations in general.
Question 2
It was felt that focusing on a small number of indicators might highlight areas that are more important to some types of practices than others. Focusing on a subset might introduce biases. Therefore, comprehensiveness of coverage is important.
It was suggested that perhaps practices themselves could add on aspirational measures. However, it would not be wise to demotivate practices at the lower end of performance that might not be able to do well on aspirational performance. Such aspirational measures might be helpful for future-proofing the tool also.
Question 3
One important consideration is the availability of data, and how this might change. In particular, changes in the use of codes by general practice clinical data systems was a concern: the advent of new Systematized Nomenclature of Medicine (SNOMED) codes to replace the current Read codes in these systems might compromise the future of the version of the tool developed here.
STAGE 1: DEVELOPING THE MEASURE
NIHR Journals Library www.journalslibrary.nihr.ac.uk 48
It would be important to review the content of the tool regularly, possibly on an annual or biennial basis, so that indicators retain usability (this would be similar to the QOF annual review, supported by NICE).
The question around whether the tool should be aimed at practice federations is a complex one. Federations are not necessarily legal entities currently, but this could change at some point. The tool should have flexibility to adapt to this. Ideally, it was felt that the tool should be focused at the individual practice level, but with the option to aggregate to the federation level.
The tool also needs to have sufficient adaptability to fit practices in different contexts: rural and urban practices are not necessarily comparable. Similarly, single-handed practices, large practices and other subgroupings also are not necessarily comparable.
To ensure that the tool can be used at the practice level, it needs continuity for tool users; therefore, it requires the sharing of information between users.
The tool also needs to be able to adapt to the integration of health and social care, including new models of care (such as those put forward in theGeneral Practice Forward View).11However, services in the GP contract need to be at the heart of such integration.11,66
Overall, for this question, the key themes emerging were the:
l need to adapt to future technological and organisational changes in the NHS
l desire to be applicable to all types of practice, but also to federations and other groupings
l need to keep the tool updated in the future.
Question 4
One key area that was seen as weak in the tool was public health. This is split between two performance areas, (1)clinical careand (2)external focus, with no clear objectives related to many aspects of public health besides the purely clinical. However, as in the workshops, participants at the meeting did not find it easy to suggest indicators of public health performance that were both available and appropriate. Public Health England data sources were suggested, as was the Public Health Observatory, but no timely, practice-level indicators were identified as being relevant for this tool.
Other areas that were discussed as possible expansion areas were screening (e.g. cervical smears),
measures for specialised populations (e.g. homeless), social care, end-of-life care, referrals and emergency admissions. Broader discussion around these did not result in any clear suggestions for different objectives besides those already included, or any firm suggestions for different indicators.
Overall conclusions from meeting
The main findings from the meeting were that there was broad general agreement that the tool (as it was at the time) was covering the right areas, although additional public health input in particular was desirable. It was also clear that the emphasis on easily collected (or preferably pre-existing) data was essential. Efforts should be made to engage practitioners in using the tool, and it should be promoted largely as a tool to help practices measure and improve their own productivity rather than a performance measurement tool. Steps should be taken to ensure its usability and relevance in the short- to medium-term future.
Online survey
The methods for the survey were given inConsensus exercise methods; the major output from this was the full contingencies for many of the indicators in the tool. These contingencies are shown in detail in
Final tool for piloting. However, the other main aspect of the survey was to provide weightings for the different objectives within each performance area, and for the different performance areas overall.
Therefore, data from the 27 respondents were collated; the relative weightings for each performance area and objective were as shown inTable 7. It is notable that, withinclinical care(which itself has the largest overall weighting, at 37% of the total), the three objectives were viewed as equally important by respondents. These weights superseded those found in the phase 3 workshops, because of the change in structure of the tool after that point (i.e. the introduction of performance areas and the splitting of clinical care into three objectives). However, the theme of the importance of bothclinical careandpractice management, which was evident from both professional and public participants in the phase 3 workshop, was retained in the consensus exercise also.
Therefore, these weightings were joined with the findings from the workshops (which balanced indicators within objectives) to create the total weighting of indicators within the tool to form an overall effectiveness index, using the following steps:
l The four performance areas were given a total number of points in accordance with the weights in column 2 ofTable 7(4920 points were available in total, so, for example, forpatient focus, the number of points was approximately 18% of 4920 = 900 points).
l Within each performance area, the objectives were given a number of points in accordance with the weights in the final column ofTable 7, using a similar mechanism.
l Within each objective, the weightings from the phase 3 workshops were used to divide the points available for each indicator.