VALIDITY OF RESEARCH

In their book Qualitative Quantitative Research Methodology: Exploring the

Interactive Continuum, Newman and Benz (1998:50-56) develop a list of 13

points for establishing the validity of predominantly qualitative research. As three of the four methods used are qualitative this research is predominantly qualitative. The list of points is as follows: 1. Neutrality; 2. Prolonged

Engagement On-site; 3. Persistent (Consistent) Observation; 4. Peer

Debriefing; 5. Triangulation; 6. Member Checking; 7. Referential Materials; 8. Structural Relationships; 9. Theoretical Sampling; 10. Leaving an Audit Trail; 11. Generalizability; 12. Negative Case Analysis and; 13. Truth Value

(Credibility). These 13 points are used as a checklist in attempting to establish the validity of the research. As Newman and Benz’s work was not published when the research was undertaken it becomes an interesting exercise to see how well the research fits their criteria.

1. Neutrality

The questions Newman and Benz suggest be asked to help establish neutrality are, questions such as: ‘Is there more than one observer? If so, is there consistency between observers?’ (ibid:51). The formation of the Action Research Group (ARG) composed of members of the EBA Working Group (management and union representatives responsible for negotiating an Enterprise Bargaining Agreement covering terms and conditions of

employment) and other interested staff and managers, was done to allow the Organization to conduct its own assessment of the Intervention. By comparing the results of the ARG with the results of my own research, as done in Chapter 8, it is hoped that the reader will be able to see that a reasonable degree of neutrality was established and verified.

2. Prolonged Engagement On-site

‘Did the author observe for long enough to get an accurate reflection of the culture or history?’ (ibid:51). Although the Intervention did not start officially

until May 1994, I had a 13 month period of discussions with the Organization prior to beginning the intervention, 12 months' intensive involvement during the Intervention, and I was still collecting data and doing minor work for the Organization 12 months after the Intervention. Surely a period stretching over three years would meet Newman and Benz’s criteria for prolonged engagement on-site.

3. Persistent (Consistent) Observation

‘Was sufficient time spent on-site to get an adequate picture of consistency of behaviour?’ (ibid). Same as for above and I am still, some seven years after the intervention, in contact with certain people from the Organization although not in any formal way.

4. Peer Debriefing

‘Did the researcher talk with any other professional to get another perspective on what he or she saw or experienced?’ (ibid). Throughout the intervention I debriefed regularly with the Managing Director and other members of an organization that I have worked for since inception. The debriefing was valuable as these people were often able to point out to me what was happening at times when I got lost, using their experience of the process in their own organization.

5. Triangulation

‘Did the researcher attempt to obtain a variety of data sources (e.g. different observers or different written histories)?' (ibid:52). As well as the nine data streams I also conducted six one-to-one consultations, of 1 hour duration, with each member of the Organization. I also surveyed 17 external agencies that frequently interact with the Organization in an attempt to validate the findings in another context.

6. Member Checking

‘That is, when a researcher returns to those people interviewed and checks to make sure he “got it right,” the researcher is member checking' (ibid).

Conducting 6 interviews with each participant provides ample opportunity for

140

member checking as does spending twelve months working on an almost daily basis in the Organization.

7. Referential Materials

‘Did the researcher use enough supportive material (e.g. documented

recordings, readings, archives or other materials that are available to others)?' (ibid). As one of the methods used was Document Studies, I think the

discussions on the Organization's documentation in chapters 7 and 8 more than meets Newman and Benz’s criteria for referential materials.

8. Structural Relationships

‘Is there a logical consistency between different data sets?’ (ibid:53). The comparing and contrasting of the different data streams through triangulation in Chapter 8 allows me to check for logical consistency.

9. Theoretical Sampling

‘Did the researcher follow where the data led?’ (ibid). In other words did I develop new data streams and follow them as they arose during the

intervention. This point is analogous to step 4, reintegration, of the IARM which allows for the adoption and integration of different data streams as they arise.

10. Leaving an Audit Trail

‘Does the researcher have good documentation, so that another researcher can easily replicate the research?’ (ibid). During the one-to-one consultations notes were written up immediately after the consultation and notes were taken during the feedback sessions conducted at the one day follow up workshops at the end of the Intervention. Other interviews on the history and culture of the Organization were either taped or noted at the time of interview. Combined with the internal and external surveys and the ARG findings, all still available in written form - as are the document studies – this would allow another

researcher to validate the research. However, whether or not the researcher would be able to replicate the intervention is another matter.

11. Generalizability

The unique nature of the Organization, the Intervention and the change agent's style would suggest that generalizability would be low. Qualitative research is not readily generalizable in the way it is in the Physical Sciences using the positivist method. However, I have now used the Complexity Theory of Change and the Intervention methodology, in the format described in

chapters 3 and above, in other organizations with good results. So if not fully generalizable it is at least transferable to different contexts / organizations.

12. Negative Case Analysis

‘Continually revising the emerging hypothesis until all known data are explained by the hypothesis is the concept of negative case analysis?’

(ibid:54). I did not do this, as it was not part of the goals of this research to run a negative case analysis. The long-standing criticism of Interventions not achieving stated aims, as pointed out in this chapter, was used as the hurdle to overcome.

13. Truth Value (Credibility)

‘[A] study has truth value to the extent that the above 12 components exist’ (ibid:56). I feel that 11 of the 12 above validity components to qualitative research have been met and therefore feel confident about the ‘… findings of the research’ (ibid).

Limitations of the Study

The anthropologist who studies a small village isolated from modern life will by his mere presence alter village life. The object of his knowledge changes as a consequence of examination. The fact that people know they are being observed can alter their behaviour (Pagels 1982:85).

Likewise my efforts to observe and measure the changes in the organization I am studying will, in and of themselves, change the organization in some way - which means that studying organizations is never going to be an exact science, which therefore should not be expected in the first place. Organization is, by its very nature, not precisely measurable.

The major limitations of the scope of this work are that it was one case study, its replicability by others is yet to be seen and someone else using a different methodology might have been able to get similar results. The main value of doing this kind of research is the insight that is generated into the process of change in human beings. And from this insight a new approach to organization change is generated.

Attention now turns to the methodology used for the Intervention.

CHAPTER 6

INTERVENTION METHODOLOGY