Document Analysis

Krippendorff (2004) provides a definition of document analysis:

‘Content analysis [document analysis] is a research technique for making replicable and valid inferences from texts (or other meaningful matter) to the contexts of their use’ (Krippendorff, 2004:18)

The above definition focuses attention on the method of document analysis. Krippendorff (2004) reminds us that a ‘text’ has no ‘reader-independent’ qualities, thus reading any text, no matter how careful and articulate the author, the reader will always incorporate a degree of conjecture and subjective interpretation; ‘the meanings

of a text are always brought to it by someone’ (p.22). Thereby Krippendorf (2004) is explicit in his contention that document analysis is a qualitative method involving interpretation of the text by the reader. Others (eg, Riffe et al, 2005), adopt document analysis as a quantitative method whereby meaning is attributed to text by counting word frequency alone (Krippendorff, 2004). That said, in order to enhance the reliability of the document analysis method the researcher notes the need to make

11

The researcher has selected the term ‘document analysis’ in place of ‘content analysis’ so that the reader does not confuse the method of ‘content analysis’ with the ‘content’ dimension of the ‘context, process, content’ framework.

114

explicit the method of making inferences from the text to allow replication (Krippendorff, 2004; Weber, 1990); this is captured in section 3.1.1.

Ultimately, the benefit of taking a document analysis method lies in the ability to sift through large volumes of data and thus analyse a much larger sample than would be otherwise possible using other qualitative methods that typically rely on small samples for analysis (Krippendorff, 2004). Stemler (2001) notes that document analysis is also useful for examining trends and patterns in documents with the additional advantage of providing ‘an empirical basis for monitoring shifts in public opinion’ (p.2). The main disadvantage of document analysis relates to problems of missing or incomplete data. In the case of this research there is an assumption that the organisation will refer to the use of Lean in their Annual Reports if they are using Lean in some way.

3.9.1.1 Document analysis: revealing the steps

This research analyses the text contained in the Annual Reports of all English general hospital Trusts in England. Identification of English NHS Trusts was via a list of acute hospital Trusts available from the national NHS website: NHS Choices

http://www.nhs.uk/ServiceDirectories/Pages/AcuteTrustListing.aspx#TrU. This

national list does not exclude specialist NHS Trusts such as those whose services are focused on the provision of cancer treatments, women’s and/or children’s health, ophthalmology etc. and thus the researcher had to manually identify and remove specialist hospitals from the list. Whilst every effort was taken by the researcher to identify all acute general NHS Trusts in England from this list, it is with regret that the University Hospitals of Leicester is known to be omitted from the dataset. This omission was accidental and due to a data inputting error on behalf of the researcher. Following identification of acute general hospital Trusts in England, data was collected using a combination of narrative analysis and Key Word in Context (KWIC) techniques to identify and collect data relating to the three dimensions of the analytical framework: context (why of Lean), process (how of Lean) and content (what of Lean). How data is collected under each of these dimensions is explained in detail below via the application of Grbich’s (2007) six questions of document analysis; table 3.4 presents a summary of how the researcher collected and attributed data under each dimension.

115

Table 3.4: Summary of data collected and attributed to each dimension of Pettigrew and Whipp’s framework of strategic change

Dimension of Pettigrew and Whipp’s framework

Data collected and attributed to dimension

Context – the ‘why’ of change Inferred from the management

commentary where the nature of the operating context is discussed in varying detail but generally conveys whether the Trust has experienced a difficult year evidenced by financial woes,

performance difficulties and/or media concern, or a successful year evidenced by good and improved performance relating to key targets, staff awards, recovery from financial debt etc.

Content – the ‘what’ of change Inferred from descriptions of projects

identified using a key word in context (KWIC) method to identify examples of Lean implementation. Eg. Reduction of waiting times in Cytology department.

Process – the ‘how’ of change Inferred from the description of Lean

implementation contained within the annual report. For example, an organisation wide approach to Lean implementation that implies Lean is implemented as a ‘system’ is inferred by a description of Lean frequently

contained within the management. This contrasts sharply with a Trust making reference to one or two isolated projects that apply Lean methods.

Annual reports have been selected as the unit of analysis because every hospital Trust in England must present to Parliament (pursuant to Schedule 7, paragraph 25(4) of the National Health Service Act 2006) a report detailing the organisation’s activities during the previous 12 months. A pilot sample of the reports found that these annual reports consistently adhere to a similar structure that provides sufficient data under each of the analytical constructs of Pettigrew and Whipp’s framework, namely: context, content and process (see figure 3.3, below for structure and content of annual reports by English hospital Trusts). Thus the annual reports facilitate a vehicle for comparison across a complete population sample. Furthermore, the process can be

116

repeated annually in order to examine the data set for trends and patterns in relation to the phenomena under study (Stemler, 2001).

Analogous to the early work of Krippendorff (1980), Grbich (2007:112) suggests that six questions should be addressed in every document analysis, below each question is addressed in turn to explicitly reveal the process adopted by the research.

1. Do you have sufficient documents to make this form of analysis useful? And which parts of these documents are to be analysed – all or part of the documents? And pertaining to what topics?

Firstly, all hospital Trusts in England present an annual report to Parliament pursuant to Schedule 7, paragraph 25(4) of the National Health Service Act 2006, thus the reports are mandatory and written for governance purposes. Secondly, all annual reports must be made available to the general public thus whilst the report must satisfy its mandatory requirements it also has flexibility in terms of style and presentation in order to appeal to the public reader.

“For each accounting year an NHS Trust must prepare and send to the Secretary of State an annual report in such form as may be determined by the Secretary of State…including information as to its forward planning, as, and in such form as, he may require.” (National Health Service Act 2006:201)

An initial sample of twenty annual reports reveal them to be vastly similar in structure and content reflecting the following guidance set out for NHS Foundation Trusts by the independent regulator Monitor shown in figure 3.3.

Figure 3.3 Guidance for Annual Report structure and content

 A directors’ report including a management commentary;  a remuneration report;

 the disclosures set out in the NHS Foundation Trust Code of Governance*;  other disclosures in the public interest;

 a statement of Accounting Officer’s Responsibilities; and  a statement on Internal Control; and,

117

 details of the Trust’s quality objectives and performance against those objectives.

(Source: NHS Foundation Trust Financial Reporting Manual (2008/09:74)

A contextualist/processualist approach to data analysis might argue that all sections of the annual reports may be deemed relevant and useful, however, in order to bound the focus of the research to the phenomena of Lean and its implementation, the researcher reads in detail the director’s report including management commentary to gather data relating to ‘context’.

Following an initial pilot sample of twenty annual reports of English hospitals the research found that the management commentary is typically provided in the report’s opening pages with a statement first by the Chairman of the Trust followed by the Chief Executive of the Trust. The length of the narrative varies in length from as little as two pages up to approximately ten pages. The narrative generally provides an overview of the past operating year in terms of the following non exhaustive list of topics that enable an interpretation of the ‘context’ element of Pettigrew and Whipp’s ‘context, content, process’ framework:

 indications of a successful/disappointing/difficult year through a discussion of highs and lows;

 discussion of any high profile incident relating to the Trust such as poor hygiene scores, a high standardised mortality ratio, medical errors and/or intervention from independent or governmental regulatory bodies;

 changes in the executive management team – new Chairman/Chief

Executive and other high profile members of the executive board;

 the Trust’s attitude to service improvement eg. a ‘turnaround’ Trust or one that claims to be at the forefront of innovation and service improvement;

 performance scores awarded by the Care Quality Commission and

118

present small/medium and large Trust of the year awards; CHKS top 40 NHS Trusts);

 financial status, eg. historical debt and/or operating deficit/surplus;

 demographic data relating to the size of population served, the density of population associated with the local areas served, ethnic diversity, and any other information/characteristics relating to the general health and wealth of the local population.

 location characteristics i.e. whether the Trust is situated in an industrial part of the country, a tourist area, rural area or city centre. Also Trust size in terms of number of staff, number of beds, budget/turnover.

In order to draw the focus of the study to the phenomena of Lean implementation without the impracticality of reading through all sections of the report, a ‘key word in context’ (KWIC) technique is used. This technique is particularly useful because the Annual Reports vary in length between 30 and 300 pages reporting diverse issues and topics and thus a lot of information is presented that is not relevant to the specific research focus. The objective of using a KWIC tool for data collection is to establish which Trusts are articulating the use of Lean, why they are using it (context), ‘what’ they are using it for (content) i.e. what tools and techniques are used and at what level (functional, department, patient pathway), and ‘how’ they are using it i.e. a project, a programme, or just a bit of an experiment or trial. KWIC employed is this way resembles Coffey and Atkinson’s (1996) hermeneutic devices, where codes (or key words) become tools to think with. Krippendorff (2004) uses a similar terminology with reference to the use of computational search techniques, choosing the term ‘hermeneutic exploration’ to emphasise that the nature of categories of analysis do not need to be fixed, rather they can evolve with the readers growing level of understanding as they become more acquainted with the context of the phenomena. The use of a computer to conduct a KWIC search allows the researcher to cut and paste the text surrounding the key word in order to maintain a transparent process of thematic analysis. The following key words have been arrived at via a combination of prior knowledge developed from the literature (alongside practical experience of Lean

119

implementation in healthcare) and accompanied by an initial pilot sample of 20 annual reports:

 ‘Lean’, as evidence of an application and/or awareness of Lean methodology;

 ‘productive’, as evidence of implementation of the national productive ward programme which is an application of the ‘5S’ technique commonly associated with Lean;

 ‘releas’, as the base form of the word ‘releasing’ and ‘release’ - part of the tag line ‘releasing time to care’ which is used synonymously with productive ward;

 ‘waste’, as possible evidence of an application of Lean methodology;

 ‘improv’ as the base form of the words ‘improve’, ‘improving’ and

‘improvement’ to highlight activities related to service improvement that may or may not be led by Lean methodology;

 ‘Rapid’ and ‘kaizen’ to identify the commonly used rapid improvement events

as a vehicle for implementing Lean;

 Program as the base form of ‘programme’ in recognition that some hospital Trust’s have taken a programme approach to service improvement that may or may not be underscored by Lean methodology;

 Project to identify the existence of a project or programme that may or may not be using Lean methodology.

The limitation of using a KWIC technique for document analysis lies in the use of the term ‘Lean’ by the hospital Trust and in the decision to articulate a Lean approach in the annual report. Thereby, it is possible that a hospital Trust is using Lean in some way but this is implicit in the report or simply not mentioned at all. In order to mitigate this occurrence, the key words identified above contain generic words that are often associated with Lean implementation such as ‘project’ and ‘program’ which may identify text that discusses improvement work in the Trust and the researcher may infer whether this ‘echoes’ Lean principles. Where a description of a project echoes Lean a further enquiry can be conducted on the corresponding Trust website to try and confirm the presence of Lean implementation in the Trust. Similarly, a website search using key words in the ‘search’ function of a Trust’s website can also

120

identify hospital Trusts that are implementing Lean but have not written about it in their reports. The website search typically picks up the use of Lean methodology in Trusts who have not specified it in their annual reports but have discussed it internally documenting implementation in archival documents such as ‘minutes of meetings’ and staff magazines. For example, there may be documents that specifically consider the implementation of Lean in the Trust following pilot projects; there may be evidence of an invitation to tender for management consultants to help roll out Lean methodology or design a programme for Lean implementation, or there might be an example of a Director recounting feedback from a Lean event at a meeting of the board and this will be included in the Trust’s ‘minutes’ which are available for public download from the hospital Trust’s website. The search function on hospital Trust websites was found to be capable of searching latent content archived on the website and identify evidence for Lean implementation. This additional KWIC approach is an important safeguard for detecting evidence of Lean implementation in the Trust when there is either no mention of Lean in the hospital report, or if there is content in the report that may imply the use of Lean methodology eg. a discussion of removing ‘waste’ from processes and/or process redesign are both strong indicators of Lean implementation but not explicit.

2. What sampling approach will be undertaken?

The sample endeavours to include each individual hospital Trust in England that offers general acute services (including A&E) to the general public. The sample excludes hospitals providing solely specialist services, for example, Birmingham Women’s hospital and Birmingham Children’s hospital are both excluded from the sample on the basis that the level of complexity may be conceived as greater in an organisation offering a myriad of services including A&E to the general public. The document analysis approach outlined under question 1 is conducted at two time intervals pertaining to the operating year 2007/08 (T1) and 2009/10 (T2). A gap of one operating year is the maximum gap feasible in this study at this time due to the nature of Lean phenomena being relatively new in healthcare and the nature of doctoral research taking place across a three year period. The gap is necessary however to provide a level of insight into the notion of ‘changing’ as dictated

121

fundamental in Pettigrew and Whipp (1991), and in parallel with the dictum of Lean

as a ‘journey’ (Radnor, 2010).

The population of acute general hospitals in 2008 was 152, in 2010, this sample had reduced to 143 following a number of hospital merges. A full list of the hospitals used in this study is presented in Volume II.

3. What level of analysis will be undertaken and what particular actions will be coded for?

The level of analysis is determined by the document, i.e. the annual report which relates to the operating activities of the Trust as a whole. Thus an organisational level of analysis is taken using the document analysis method. The actions to be coded for are guided by the analytical constructs of Pettigrew and Whipp’s context-process- content model. As described under Grbich’s (2007) question 1 above, the ‘context’ dimension is mostly satisfied through interpretive analysis of the management commentary or narrative provided in the opening of the annual report coupled with more objective demographic data identified from the annual report and the Trust website. Other aspects of context that are coded for include external data such as the Strategic Health Authority (SHA) that the Trust operates under and any physical attributes that are mentioned in the Annual report or on the Trust’s website, eg. a popular tourist location or situated near major airports and motorways or situated in a heavy industrial area for example. Such attributes can have an impact on the type and level of demand experienced by the hospital Trust. Also under context are internal issues relating to more objective data such as Foundation Trust authorisation, performance scores and movement of Chief Executives.

Data relating to the Process and Content dimensions are derived using the KWIC method using key words as codes described and identified under question 1.

4. How will the protocol and/or codes be generated? Will you seek these from the database via preliminary data and thematic analysis or will you impose a pre decided (a priori) coding frame derived from the literature and your own experiences in this field? And if the latter, what inclusion and exclusion criteria will you use to develop pre-decided codes?

122

The analytical framework guides the data collection under three themes: context, process and content. The key words are the codes in the data and these are derived from a combination of the literature, preliminary analysis of a sample of 20 annual reports and my own experience observing improvement workshops in hospitals. The data collected under the three themes is ‘cut and paste’ from the annual report document into the tabular proforma for thematic coding, interpretive analysis and categorisation. The copied section is not limited in uniform fashion to ‘n’ number of lines either side of the key word, rather all of the surrounding text is copied as deemed relevant to the dimensions of the analytical framework and the phenomena of Lean.