Refining the Improvement Type Measure

To reflect the methodological move towards verbal explanation rather than paper- based description of improvement work, the data-gathering process for the Improvement Type Measure was incorporated into the semi-structured interview phase of the project. Some final refinements were made to the ITM prior to commencing the semi-structured interviews.

Firstly, the measure about the reasons or drivers for change (the voluntary / compelled dimension) was re-considered and removed. The pilots had not provided data to suggest that this was a discriminating factor in the way improvement work was implemented. Some feedback from the pilots had highlighted this item as being a ‘red herring’ and the researchers also had reservations about its value. The researchers decided that the reasons for an improvement were part of the context for that change, and that the study’s focus was on implementation within the given context. Whilst acknowledging the significance of contextual factors, it was beyond the scope of this study to examine contextual factors in specific terms.

Secondly, a factor which emerged as warranting more attention was the sustainability of the changes made to services. The term ‘sustainability’ has become common parlance in relation to organisational improvement. Within improvement science and process improvement fields, sustainability is defined as,

‘when new ways of working and improved outcomes become the norm… not only have the process and outcome changed, but the thinking and attitudes behind them are fundamentally altered and the systems surrounding them are transformed as well.’ (NHS Institute 2011, p.4)

In recent years, academic studies have been devoted to understanding why improvement processes are often implemented successfully in organisations, but are difficult to sustain over time. (e.g. Bateman 2005).

In the context of the NHS, the difficulties of improvements becoming mainstream is acknowledged as a continuing challenge. The NHS Institute for Innovation and Improvement has an entire workstream dedicated to supporting sustainability in service improvements, describing successful organisations as those which,

‘can implement and sustain effective improvement initiatives leading to increased quality and patient experience at lower cost’. (Online source, NHS Institute 2011a)

In their account of what had been learnt about service improvement in the NHS, Maher & Penny (2005) describe sustainability as,

‘being able to hold the gains made during the improvement initiative, evolving them as required and definitely not going back to the old ways of working.’(p.94)

This captures some of the meaning intended by the researcher in introducing sustainability as a dimension for the ITM. More specifically, the item was intended to relate to the influence of the improvement leader, whereby a sustainable improvement would be one where the leader could leave the organisation and the improvement made would be sufficiently embedded into the way of doing things that it would continue even after they had left. In other words, the improvement was not dependent on that individual leader continuing to actively support or promote it. An item was therefore added to capture data about the extent to which the improvement work was a ‘one-off’ or was becoming embedded into the way the organisation works i.e. was it dependent on the individual leader or ‘champion’ or did it become part of ‘the way we do things around here’?

Thirdly, early discussions about the ITM dimensions had included debates about the extent to which the Focus and Level of the quality improvement work could be differentiated or conflated, and this issue needed to be resolved. An assumption existed amongst some members of the team that an improvement occurring at a local level, led by someone in the lower hierarchical levels of an organisation, would require simpler, and possibly fewer, leadership skills than a change being led by someone senior in the hierarchy, attempting to change things more strategically. Other members of the team remained unconvinced about this issue.

It was decided that data gathered from the study would help to illuminate this issue and to indicate how significant Level and Focus were in terms of their links to leadership for NHS improvement. Accordingly, the two separate dimensions of Focus and Level were retained within the ITM, to be tested against the data gathered.

Fourthly, a decision was made to extend the scale for each dimension from a 3–point scale (as outlined in Figure 2) to a 7-point scale. The team’s increasing familiarity with the nature of the improvement work undertaken by THF Award Holders led to a concern that there would be a large degree of clustering of ratings towards the centre of a 1-3 scale, resulting in many ratings of 2. This could potentially obscure differences in the types of improvement work, and make it more difficult to uncover relationships and links between datasets. The move to a 7-point scale was therefore designed to allow a greater degree of differentiation between the various quality improvement initiatives being carried out. Detailed descriptors were established for the low, mid and high points on this scale (1, 4 & 7.) A worked example of how ratings were established using these descriptors is provided in Section 5.4.1. These descriptors proved effective in enabling members of the research team to reliably rate the types of improvement work. Given this efficacy, similar descriptors for ratings 2, 3, 5 & 6 were not pragmatically required for the purposes of the study. However, for wider utility beyond an ‘expert’ group of users, detailed descriptors at every rating point would be necessary. This is considered further in Section 7.8.

The lengthy, iterative process of developing an approach to categorising different types of NHS improvement illustrates the significance of this phase of the work as a basis for the rest of the study. The final measurement dimensions were arranged to

form broad headings of Focus, Level, Process and Intended Impact, summarised into what was named the Healthcare Improvement Typology (Figure 4).

The Healthcare Improvement Typology reflects the changes to the dimensions made in response to the literature, the piloting of the measure and the internal reliability testing within the research team. It enabled each piece of improvement work encountered during the data-gathering to be classified, with a 4-integer rating (e.g.F2L4P3I5, abbreviated to 2435). This provided the working taxonomy for the study, and formed the basis of analysis and correlation to leadership behaviours.

Degree of Complexity Dimension 1 4 7 FOCUS The improvement is aimed at a defined group of people and is limited to a single clinical condition or one aspect of a clinical pathway.

The improvement is aimed at a wide group of people with a range of clinical needs.

The improvement is intended to benefit unlimited numbers of people with unlimited clinical needs.

LEVEL

The improvement is focused within a single ward, department or general practice.

The scope of the improvement covers several departments or care pathways within a single health economy.

The improvement covers several national and/or international agencies or organisations.

PROCESS

The change involves small improvements to existing practice. It only involves influencing one or two specific, identifiable individuals, and the task involved in this is extremely easy.

Some aspects of the change involve different ways of doing or thinking about things. Influencing is both direct and indirect, involving identifiable individuals and identifiable groups of people. Some of this influencing is problematic.

The change is entirely innovative, with completely new ways of doing or thinking about things. It involves influencing a range of people so diverse that it is virtually impossible to define them all; a task as complex and difficult as it could possibly be.

INTENDED