Reliability and validity

The research undertaken must prove to be trustworthy. Merriam (1998:199) states that researchers and others must have confidence in the conduct of the investigation and in the results of any particular study. I will briefly define internal validity, reliability and external validity and then go on to explain the specific steps I took to ensure that the data collection, analysis and interpretation were reliable and valid.

According to Merriam (1998:199) internal validity is linked to reliability in that others must have trust or confidence in the rigour with which the study has been conducted and the results or findings of the study. In other words the results must show ‘dependability’ or ‘consistency’.

Merriam (1998:206) describes the traditional definition of the term ‘reliability’ as an oddity in qualitative research in that replicating a qualitative study will never produce the same results. Instead she replaces the term ‘reliability’ with Guba and Lincoln’s coining of the terms ‘dependability’ or ‘consistency’(ibid p.206) regarding the results gained from the data. Using these terms means that the findings need not be the same for a similar study. Instead, the results need to be consistent with the data collected.

Guba and Lincoln (cited in Merriam, 1998:206) state that before the external validity of a study is discussed it must have internal validity. In investigating the process of data collection, analysis and interpretation for external validity the reader or user of the study must decide what is appropriate and fitting for their own situation (Merriam, 1998:211). In other words, the reader or user of the study must determine what he or she can and cannot apply to his or her particular situation.

3.7.1 Steps taken to ensure internal validity

I took the following steps to ensure internal validity in this study:

a. Member checks - I asked the respondents to approve the typed interview transcripts and then also the tentative interpretations of the findings in terms of what they saw as the main themes (Henning, Van Rensburg and Smit, 2004:149). In other words, I attempted to capture the meanings and perspectives that respondents hold of themselves and their life world by including their viewpoints as part of my analysis. In this way, according to Lincoln and Guba (1985:296, in Eisenhart and Howe, 1992:651) the chances of the respondent categories and not my perceived or assumed categories dominating the findings would be increased.

b. Peer examination – to achieve this I asked three of the respondents (one from each faculty) to comment on the findings of the research as they emerged (Merriam, 1998:204).

c. Triangulation – I used two methods of analysis to confirm the emerging findings (Merriam, 1998:204). These included narrative analysis and the constant comparative method of analysis. I relied on the “holistic understandings” gained through narrative analysis, as termed by Merriam (1998:204), to gain reasonable explanations about the personal and social identity positioning of lecturers.

3.7.2 Steps taken to ensure reliability

Due to the fact that I was interviewing individuals who are influenced by the specific university context in which they work and are influenced by their own cultures and norms and society in general it is obvious that the same results would not be found in a different context. It stands to reason that lecturers in different contexts might behave differently and express different opinions despite similar conditions with the same interviews at other higher education institutions.

I took the following steps to ensure reliability in this study:

a. Investigator position – I have explained my assumptions and the theory behind the study, my position versus the position of those lecturers’ being interviewed as well as the basis for selecting respondents along with a description of them (Merriam, 1998:206-207).

b. Triangulation – as discussed above with regard to internal validity.

c. Audit trail – Terre Blanche and Durrheim (1999:426) state that interpretive research often has many illustrative quotes or anecdotes and as a result researchers often “feel their way” to their conclusions or rely on personal experiences and background knowledge of the field. However, Miles and Huberman (1994:427) advise that the audience of any research report should be provided with an audit trail so that the findings of the research can be authenticated. This involved me providing a clear outline of the research process in terms of the interpretive framework employed.

My audit trail will consist of the interview protocol (Appendix B), one sample interview transcript (Appendix A) and the consent form (Appendix C) which each respondent signed prior to the interviews being conducted as well as a table of codes collected and how these were constantly compared to reach the findings of the study (see chapter 4). These provide proof that there is a clear description of processes being followed in the collection of data.

3.7.3 Steps taken to ensure external validity

I took the following steps to ensure external validity which are suggested by Merriam (1998:211):

a. Rich, thick description – I converted thin descriptions or raw data into thick descriptions by providing an interpretation of the facts and content in terms of the theoretical framework that locate the study (Henning, Van Rensburg and Smit, 2004:6). The raw data was clarified with discussion and analysis and enriched with

explanation and argument (ibid , 2004: 85) by focusing on the research questions, the purpose of the research and the specific purpose of the interviews. This was done by presenting the “thick description” with ample empirical evidence as well as supporting this with a strong theoretical base (ibid, 2004: 7).

b. Typicality or modal category – LeCompte and Preissle (in Merriam, 1998:211) describe this as the typicality of individuals, so that other users can make comparisons in terms of their own situations. In this study my sampling of lecturers was purposeful. These lecturers represented typical lecturers in that teaching and research formed the basic requisites of their job descriptions. I am of the opinion that a similar study would be applicable to other universities, due to the purposeful sampling of lecturers.

c. Comparability – this study would also be comparable to other universities in that the units of analysis, concepts generated, population, characteristics and setting are clearly stated. Le Compte and Priessle (1993) state that “the degree to which the components of a study, including the unit of analysis, concepts generated, population characteristics, and setting, are sufficiently well described and defined that other researchers can use the results to compare to other studies addressing related issues.”