A Matrix Approach

In order to accommodate the multi-faceted nature of the service improvements being studied, the researcher explored the concept of a matrix typology, whereby different levels of complexity could be represented through a range of dimensions, making up an overall typology framework. The next stage of work was to identify the most appropriate dimensions to include in such a matrix typology.

Based on the initial analysis of the documentary data, and from extensive experience of leadership development activities with a focus on improvement work, an initial set of dimensions was considered:

Focus what is the target group and scope of the improvement?

Level where, in structural terms, in the system or organisation is the improvement taking place? e.g. front-line, middle/ operational, top/ strategic, intra- / inter-organisational, national, international

Process how is the improvement being led? e.g. methods, tools, approaches

An initial 3-dimensional taxonomy model was mooted, based on Focus, Level and Process, with progression along each axis implying an increasing complexity, as shown in Figure 1.

The ‘Focus’ dimension was originally intended to encompass the number and type of patients involved in the quality improvement, and the nature of the intervention. The ‘Level’ dimension covered the level of the organisation at which the initiative was taking place, from single departments or intra-organisational linkages, through inter- organisational relationships, right through to national or international working. The ‘Process’ dimension encompassed aspects such as the number of stakeholders involved, degrees of resistance, and complexity of the change management process itself. The 3D model (Figure 1) was piloted in various parts of The Health Foundation, including with the Leadership Development Consultants and staff from within the Leadership Programme. There was an intuitive agreement with the basic model, and a high level of interest in the multi-faceted way of capturing diverse types of improvement work.

When referring to the quality improvement literature to inform the development of these dimensions, the most pertinent recent contribution to the literature was Walshe’s (2007) discussion of the need for theory-driven evaluation of quality improvement. His analysis defined four main variables of quality improvement as being:

Content the situation, setting or organisation in which the QI intervention is deployed;

Context the nature or characteristics of the intervention itself;

Application the process through which the intervention is delivered;

Walshe’s categories resonated to some extent with the Focus, Level and Process dimensions from the research team’s internal analysis. Walshe’s ‘content’ variable mapped onto the Focus dimension, with less emphasis on identifying the organisational setting, and more attention paid to the specific areas within the organisation which were subject to improvement.

The ‘Level’ dimension was not explicit and separate within Walshe’s discussion, but instead was conflated into the ‘content’ category. The research team decided that it seemed relevant to retain a separate dimension to capture data about ‘Level’, as there was such a range of data about this from the early documentary trawl of application forms and end of award reports. For example, some improvements were happening very directly at the front-line of service delivery, such as in wards, operating theatres and clinics. Others were much more organisation-wide, or beyond a single organisation. This differentiation seemed important to capture via the typology, in order to investigate its potential relevance to how improvements are led.

Walshe’s ‘context’ and ‘application’ variables both focused on how the improvement was delivered. This was similar in meaning to the ‘Process’ dimension of the typology, which aimed to consider the way the improvement was led. The final variable identified by Walshe was ‘Outcome’. An important aspect of the scoping of this research study involved the extent to which the outcome of improvement work was relevant to the research. As explained in Section 4.5, this study did not aim to specifically measure the outcomes of improvements.

The next stage of developing the Improvement Type Measure was to decide how to categorise examples of improvement against the proposed dimensions. Using the idea of a matrix as a basis, numeric values 1-3 were added to each dimension, to allow them to be compared and to some extent, measured in relative terms. Each example could therefore be categorised with a rating such as Focus 1 Level 2 Process 2 (F1L2P2); F3L3P3 etc. This matrix, shown in Figure 2, was known as the FLP matrix, indicating its three dimensions of Focus, Level and Process.

1 2 3

FOCUS

Single patient group Single intervention / outcome

Multiple patient groups Multiple interventions / outcomes Indeterminate patient groups Indeterminate interventions/ outcomes LEVEL Within a single organisation Across several organisations Beyond inter-organisational e.g. regional, national or international

PROCESS Defined and simple Defined and complex Ambiguous

Figure 2. Draft FLP Matrix Measure of Improvement

This stage of typology development was highly iterative, with various members of the research team concurrently refining and testing different aspects of its reliability, utility and validity. Each of these aspects is detailed in the next sections.