Chapter 7: Discussion and Conclusion
7.2 Research Question 2: To what extent are the quantitative data generated during Productive
Chapters 4-6 used mixed method studies to examine quantitative data generated during the
implementation of Modules 1,3 and 4, and the qualitative data from interviews about the modules. In answer to research question 2, the studies found that there were many ways in which the quantitative data reported lacked meaning and were unreliable. For example, data from the WOWE ‘5S Inventory Sort’ in Chapter 4 indicated that approximately £42,500 had apparently been ‘saved’ by the organisation, however there were many aspects of the implementation method that made this figure unreliable because the cost of products were not validated, and qualitative data found that staff were not always truthful about the data submitted (see page 115). Equally, the meaningfulness of the figures was brought into question as the value included items that had expired or were discarded, so was not a true saving (page 123). Wright and McSherry (2013) categorised most of the Productive Ward articles in their systematic literature review as ‘Anecdotal’, which they define as “Journalistic articles which tend to briefly explain the
structure of the programme and present headline findings such as a significant improvement in a specific performance measure,” (Wright and McSherry, 2013, p. 1364). The ‘saving’ of over £42,500 was published in the organisation’s internal progress report, and had the organisation been asked to provide information for an ‘Anecdotal’ type of report by the NHSI, this figure would have probably been provided. In addition, it was found that although the figure saved by the Swap Shop initiative was substantial, there were some ‘hidden costs’ that were not reported in the internal progress report (page 123), which again if published anecdotally, would have been misleading.
In Chapter 5 the results of the ‘PSAG Screen’ timed exercise indicated a 62% reduction in the time taken to find patient information in SystmOne. Again these were seen to be unreliable due to the method used (e.g. many staff timed themselves, see page 132) and lacking in meaningfulness (e.g. as actual benefits to staff in regards to time savings would be spread out over a long period of time, see page 151). However, although the design was flawed, it did identify the lack of a performance curve in the ‘PSAG Screen’
condition which otherwise may not have been apparent (see page 136). Again this could have been reported in an ‘Anecdotal’ report where the processes behind the figure would not need to be explicit and issues of meaningfulness not explored. In addition, previous research on PCS has also found relatively small amounts of time saved (e.g. YHEC and NHSI (2010) suggested that time savings made through the programme could increase the duration of patient visits from 28 to 35.7 minutes on average). These small savings may mean that staff may only notice the cumulative benefits from the programme, and may not perceive a benefit from the time saved by individual exercises. As also proposed in Chapter 3 (see page 93), this indicates that in implementations of programmes such as the Productive Series or Lean, there is likely to be a need for many initiatives to be carried out in order for staff to perceive the innovation’s
‘Relative Advantage’ (Greenhalgh et al., 2004; Rogers, 2003) from the cumulative effect.
The quantitative data collected during Module 4 (see Chapter 6) indicated that services spent an average of 36% of their working week in contact with patients (face to face or via telephone). There were
numerous issues with the reliability of this figure (e.g. the duration of leave or training in some instances
was duplicated during data collection) and meaningfulness (e.g. the inclusion of administrators’ time).
Again this highlights the issues that might be associated with figures published in ‘Anecdotal’ reports but are not made explicit. This also raises the question of whether Patient Contact Time is a meaningful measure at all. Although one of the main premises of the programme is to increase time spent with patients, the programme authors emphasise that this should be an increase specifically in Patient Facing Time. This might be identified as a tension between the PCS Programme Theory and community practice, as the PCS PM recognised that a lot of work in the Community takes place over the telephone, and so the definition was changed to Patient Contact Time. Although the programme authors give examples of innovations that increase productivity by treating patients remotely (Module 8 includes a best practice example of a telehealth system being used, see NHSI, 2010b, pp. 62-63), they do not clearly explain how or whether this remote type of Patient Contact Time should be acknowledged, even though it is likely to impact on productivity in ‘narrow’ terms (page 175), as it reduces the travel required to visit the patient.
In addition, an increase in Patient Contact Time in itself may not increase productivity if the intervention carried out during that time is ineffective. This supports the argument to measure care based on patient outcomes rather than staff activity (see Vallance-Owen, 2015), although arguably the ideal to work towards is to maximise both the time spent with patients and the effectiveness of interventions. As the programme authors try to address both of these aspects, in theory the programme as a whole should be beneficial to patients and staff. In practice, although patient experience data was collected during Module 8, no data relating to patient clinical outcomes was collected, so it was not known whether the programme had contributed to an increase in the effectiveness of interventions. Implementations of efficiency
programmes in healthcare should therefore ensure that key clinical outcomes are measured, alongside any measures of Patient Contact Time, to ensure that clinical effectiveness is not being compromised by
‘Leaner’ processes.
The findings from these chapters highlight the importance of carrying out research on change or improvement innovations in healthcare so that the processes behind outcome figures are made explicit.
The findings also highlight the need of increasing access to this research. The ‘Anecdotal’ reports of these initiatives often served as advertisements for the programme (see page 13) however it would be more helpful for healthcare staff to be able to access not only the anecdotally reported outcomes of the programme, but the learnings and recommendations from the implementations. Reports including these do exist (for example see Bloodworth, 2009), but appear to be rare. Therefore the implementation and publication of this research (e.g.Bradley and Griffin, 2015) makes a valuable contribution to the healthcare field.