Validation of category relationships

The inductive analysis of interview data (my first unit of observation) has resulted in identification of 15 significant relationships between selected categories listed earlier in Table 4–5 and 4.6. These relationships are expressed in terms of my unit of analysis, the implementation of an ERP project.

I have now used my second unit of observation, project documentation provided by interviewees, in order to validate these 15 relationships. This was only possible where relevant documents were provided; clearly documents relating to Y2000 compliance were not provided because of the elapsed time since completion of this category of project.

4.3.1 Validation by use of ERP project documentation

The 20 key informants interviewed provided a wealth of project documentation from ERP projects in which they were currently engaged (in the case of both the business project managers and engaged consultants). A wider range of documentation was provided by the ERP consultants, who generally had experience of over 10 separate company ERP implementations. These documents aligned longitudinally with the main implementation processes, from initial business cases and implementation plans to project meeting notes, progress reports and finally post-implementation reviews.

Bryman (2008) has proposed three methods of interpreting documents; qualitative content analysis, semiotics and hermeneutics. Qualitative content analysis comprises a searching out of underlying themes in the materials being analysed . Altheide (1987 outlined an approach termed ethnographic content analysis where the researcher is constantly revising the themes or categories distilled from the examination of documents. Both Bryman and Altheide emphasise that qualitative analysis involves applying predefined categories to the analysis . This summarises my

approach to using project documents as validation of predefined category relationships.

The project documents that were my second unit of observation were not subject to the inductive coding processes that I have detailed earlier in this chapter. These documents were used to support relationships between predefined categories that were obtained from interview data, my primary unit of analysis. In this sense, project documents were used as a method of data triangulation, the validation of established category relationships by extracts from secondary data. The project documents were not subject to the coding processes described earlier because they were regarded as secondary data and also because the sheer volume of documents handed to me by interviewees would have made a full inductive analysis impractical (in one interview I was given a book).

In summary my approach to the analysis of documents was as follows:

(1) project documentation was sorted into folders that followed the chronology of the enterprise systems lifecycle, from business case and other planning documents through to project implementation documents and then post-implementation documents.

(2) the coding of interview data provided selected categories for further analysis to determine relationships that were likely to most revealing in the context of my research questions. These 15 key relationships have been described as pre-implementation and post-pre-implementation relationships (see Table 4–5 and Table 4–

6).

(3) project documentation was then selectively analysed to identify phrases or paragraphs that correlated to the particular category relationship under study. So, for example, where the category relationship was R-6, between C (business cases) and C38 (metrics used to gain approval), documents relating to business cases were examined to identify content that supported the above relationship. Post-implementation review documents were studied likewise for document support for relationships R-10 to R-12 between retrospective reviews and other related categories.

This provided the primary method of integrating interview and documentary data.

4.3.1.2 Limitations of project documents provided

Bryman has written people who write documents are likely to have a particular point of view they want to get across . The author continues members of different groupings (within a business organisation) expressed through the documents certain perspectives that reflected their positions in the organisation . The majority of the informed participants were management consultants engaged by the ERP adopting business. This grouping primarily provided documentation that was jointly prepared by the consultant and the client. )n many cases the documents were part of business proposals to obtain consulting work on a competitive basis and included estimates of the business benefits of the ERP implementation project (which almost always exceeded the costs of the project, including the consultant s fees . Clearly it was in the interests of this grouping to stress the business benefits, whether strategic or operational, of the project and their associated involvement. Likewise, where external post-implementation reviews of an ERP project were completed these often recommended the further use of external consulting support. As a result, caution was necessary in interpreting documentation provided by these practitioners because of the above factors.

Other points should be emphasised regarding the scope and use of project documents provided:

(1) The researcher made no request, either before or during interviews with informants, for supporting documentation. The informants freely provided documents that they felt would be helpful to the researcher, rather than being asked to provide documentary evidence of responses to the protocol.

(2) Informants did not provide documents that supported all the 15 category relationships; for example, business cases that were driven by Y2K compliance were not supplied for obvious chronological reasons.

(3) Documents that related to the project implementation (Stage 2) were acknowledged but not subject to the same level of analysis as other documents because this stage was largely outside the scope of my research question and identified category relationships. However, extracts from Stage 2 documents have been included in Appendix 5.