What did we find?
For this indicator we looked at different ways of displaying response time performance by calculating percentile response times and different variations on mean time (see Table 7). Mean values include all cases but can be skewed by small numbers of very high or low values. We looked at alternatives such as
DOI: 10.3310/pgfar07030 PROGRAMME GRANTS FOR APPLIED RESEARCH 2019 VOL. 7 NO. 3
© Queen’s Printer and Controller of HMSO 2019. This work was produced by Turner et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
trimmed and Winsorised means, which treat extreme values differently and may provide a more useful indicator. We calculated response times for different types of call to illustrate how this could be used for within- and between-service comparisons, for example urban versus rural times using lower super output area data and different categories of urgency. These are presented in detail in Appendix 2. An example is given in Table 12.
The tables have shown that displaying mean and percentile response times can provide a useful indicator of response performance that can highlight differences in service provision; for example, it can be used to compare performance in urban and rural areas or at different times. These are important when considering resource management and provision of an equitable service. It also provides a picture of variation across the whole response time spectrum, not just a proportion of calls. Displaying percentiles adds transparency so that longer as well as shorter response times are visible.
The different approaches to calculating mean and percentile response times by displaying standard mean (using all values) and alternative methods for managing extreme values (trimmed and Winsorised means) present different pictures. In particular, the differences are more marked for call categories with longer expected response time standards (e.g. 30 minutes) than those with short expected standards (e.g. 8 minutes). The larger mean values for the standard mean reflect an important issue in ambulance service delivery in that these take into account the very long waits that some patients experience. It is also possible that some of the very long response times reported may be the consequence of data inaccuracies rather than a real extended response time and, therefore, present a distorted picture that could lead to erroneous assumptions about performance. Trimmed or Winsorised means allow these potential data problems to be accounted for. All of the methods for calculating means, and the corresponding percentiles, have their own advantages and disadvantages and the choice of which to use will be as much a feature of what they will be used for and who by. For example, if reflecting long waits is important then standard means may tell a better story, but for frequent monitoring of overall patterns or trends in performance, use of an alternative may provide a more consistent approach that smooths out the effects of short-term variation, such as call demand spikes or bad weather, which inevitably affect response performance.
We presented the alternative ways of describing response times to our PPI reference group. From a user perspective, they found the median and 90th percentile measures most useful as these could be interpreted as, respectively, half of calls or 9 out of 10 calls being responded to within a given time. They thought that presenting actual times was more informative and transparent than percentages within a specified time, as the latter option obscured what was happening to calls outside the target time.
Indicator 4: proportion of decisions to leave a patient at scene (‘hear and
treat’ and ‘see and treat’), which were potentially inappropriate
What did we find?
For this indicator (see Table 8), we examined four variables and included three in the final multivariable model. There was a non-linear relationship between re-contacts or admission and age with the probability of re-contact increasing with age up to the 70–80 years age group and then the probability decreased as age increased further. Sex and reason for the call were also associated with re-contact. Reason for the call was characterised by 10 broad groups: abdominal pain, breathing problems, cardiovascular, falls, fitting, injury, psychiatric, sick person, unconscious and other conditions. There was a small proportion of cases where no problem was recorded (e.g. transfers from NHS111).
We also calculated standardised rates per 100 calls by CCG and time of call to test if the model could be used to assess performance between areas and at different times. Although the model did not meet the statistical threshold for a good case-mix adjustment model, the results suggested that case-mix adjustment improved the usefulness of the indicator by controlling for factors that could make interpretation of crude
measure standardised rates between areas and over time and case-mix adjustment provided a potentially more sensitive measure. The re-contact or admission rate for non-conveyed patients ranged from 5 to 10.2 per 100 calls across 22 CCGs. This means that between 90% and 95% of decisions to not convey a patient to hospital were appropriate.
Sensitivity analysis showed that standardised rates were smaller if untraced calls were included. Better information enabling more calls to be traced would improve the model. A limitation is that we cannot know what proportion of untraced calls had a re-contact or died and how many were untraced simply because their problem was resolved by the ambulance service and they had no further contact with the health service within the time frames of our linked data. This is a particular issue with‘hear and treat’ calls. Some re-contacts may be justified as the condition has worsened but the original decision was appropriate but the indicator is useful as we would expect the standardised rate to remain stable or improve over time. If it becomes worse, or there are substantially different rates between services, this would indicate a problem.
This indicator is linked to the next indicator (5– potentially unnecessary conveyance).