Reliability of semi-structured interviews

Lincoln et al. (1985) maintain that reliability in qualitative research should be viewed as

consistency rather than replication, that is, rather than demanding that outsiders come up with the same results, a researcher wishes outsiders to concur that, given the data collected, the results make sense. Thus the results are consistent and dependable. The question is therefore, not whether the results will be found again but whether the results are consistent with the data collected. This supports Merriam (2009) who states that replication of a qualitative study will not yield the same results, however, this does not discredit the results of the original study. Several interpretations of the data can be made, and all stand until directly contradicted by new evidence. Thus, if the findings of a study are consistent with the data presented, the study can be considered dependable.

Hesse-Biber et al. (2011: 53) suggests that researchers use Gay and Airasian’s guidelines (2003: 536) for evaluating reliability in qualitative studies which consists of:

A description of the setting of the interviews and the researcher’s relationship to the participants;

o This description formed part of the research journal. This supports Merriam (2009) who maintains that good qualitative research gets much of its claim to reliability from the researcher’s ability to show convincingly how they got the data and how they built confidence that this was the best account possible. Furthermore, Bird (2005: 235) states that there is little value in a face-to-face interview unless the researcher added “observations and interpretations” of the event which increases the rigour of the research.

All field documentation should be comprehensive, fully cross-referenced and annotated, and rigorously detailed and form part of the research journal;

Observations and interviews must be documented using multiple means such as written notes and audio recordings;

o The researcher kept a research journal and recorded the interviews which were then transcribed.

The interviewer’s training should be documented;

o In this case, the researcher is a trained interviewer who has completed training in interview techniques. The researcher, in the course of her work as a writer, has conducted interviews with a range of people, from general workers to people who work at executive management level.

Key informants must be fully described;

o This aspect formed part of the research journal.

The construction, planning and testing of all instruments must be documented; o This forms part of the methodology section of this thesis.

Sampling techniques must be fully documented and need to be sufficient for the study. Internal reliability

Reliability refers to the internal consistency of field observations. The researcher needs to ask such questions as:

Whether the collected data was reasonable? and

Does the data fit together? That is: does the data add up and is there consistency in the observations over time and in different social contexts?

Morse, Barrett, Mayan, Olson and Spiers (2002) maintain that methodological coherence is an important factor in ensuring the internal reliability of a research study. Thus the aim is to “ensure congruence between the research question and the components of the method” (Morse et al., 2002: 18). The question must match the method which must in turn match the data collected and the analytic procedures used to interpret the data. In this research study, the researcher felt that the research questions were best answered by using a mixed method approach consisting of an initial survey and followed by semi-structured interviews. In addition, the various data analysis techniques used in this study are consistent with the type of data gathered.

External reliability

External consistency refers to verifying or cross-checking observations with other divergent sources of data, that is, triangulation. Thus it is important to look for evidence that confirms and supports findings. Morse et al. (2002: 18) state that it is important for the researcher to “think theoretically” as ideas that emerge from the data can be reconfirmed in new data which can result in new ideas that “must be verified in data already collected.” Thus the ability to think theoretically requires a combination of micro and macro perspectives, “an inching

forward without making cognitive leaps, constantly checking and re-checking, and building on a solid foundation”(Morse et al., 2002: 18).

The qualitative component, the semi-structured interviews, required a more inferential approach than the surveys and one which relied on the researcher’s interpretive ability and judgment. Specific procedures were put in place to ensure that patterns and themes in the data were confirmed. These procedures included inter-rater reliability which was established by allowing two different colleagues to examine the data collected from the interviews in order to establish agreement regarding the selection of the interview themes (Seliger et al., 1989: 185-6) and re-grounding whereby the researcher returned to the interview data a second time in order to compare the patterns and themes with those obtained from the first viewing so as to

confirm initial conclusions (Seliger et al., 1989). The interview data is presented in terms of themes that emerged from the interviews supported by quotes taken from the transcripts. The themes are limited to those that pertain to the research questions.

The interview themes were open to a more subjective interpretation than the results of the MBI-ES. The researcher used the techniques of inter-rater reliability and re-grounding to increase the level of internal reliability (Seliger et al., 1989: 186). The data which was obtained from the interviews revealed patterns and themes, arising from interviews about stress, support structures and coping strategies, which would be common to any sample of TESOL teachers working at private language schools in South Africa. If an independent researcher interviewed a similar sample of TESOL teachers using the same questions, similar themes and patterns would emerge.