Reliability and validity

To understand the meaning of reliability and validity in research studies, it is necessary to consider the interpretation of researchers from different perspectives (Golafshani 2003:600).

The ontology – dealing with the nature of being – of postpositivists can be portrayed as critical realism, “real reality”, but is only imperfectly and probabilistically apprehensible

(Lincoln & Guba 2000:195-196). According to the postpositivist paradigm the extent to which results are consistent over time and represent the total population included in a research study accurately is referred to as reliability. If the results of a research study can be reproduced under a similar methodology, the research instrument is considered to be “reliable”. Validity, on the other hand, is used to determine whether the research truly measures that which it was intended to measure and also how “truthful” the research results are (Joppe 2000:1).

Researchers state that historical research and descriptive studies are more concerned with finding new explanations and interpretations of existing information than collecting new information (Welman & Kruger 2001:180). In historical research, in view of the uniqueness of historical incidences, reliability and validity do not enjoy such a high priority as “replicability” (the extent to which measurement techniques constantly

construct the same results). The scientific requirement of replicability does not refer to a repetition of events, but the ability to duplicate procedures, analyses and conclusions. Internal validity reflects the extent to which available sources make it possible to rule out alternative explanations or conclusions. External validity reflects the effectiveness with which the findings enable the researcher to predict the course of events in other times and places.

The use of primary sources is an important principle of historical research as the transfer of information from one source to another may be inadvertently distorted. In historical research, information from primary sources is usually obtained through purposive or snowball sampling and unstructured or semi-structured interviews. Information from both primary and secondary sources should therefore be subjected to stringent criticism. External criticism is directed at the genuineness and authenticity of the source whilst internal criticism reflects the accuracy or credibility of the contents of the source. The information that stood the test of internal and external criticism should be synthesised and interpreted in an attempt to propose explanations and correlations. Fundamentally such explanations amount to interpretations of the evaluated information by means of inductive logic.

While the terms reliability and validity are considered to be essential criteria for quality in quantitative paradigms, the terms credibility, neutrality or confirmability, consistency or dependability and applicability or transferability are considered to be the essential criteria for quality in qualitative paradigms (Lincoln & Guba 1985:300).

The term “dependability” in qualitative research closely corresponds with the notion of the term “reliability” in quantitative research (Lincoln & Guba 1985:300). The traditional

quantitative view of reliability is based on the assumption of repeatability. The idea of dependability, on the other hand, emphasises the need for the researcher to account for the ever-changing context within which the research occurs and is responsible for

describing the changes that occur in the setting and how these changes affected the way the research is approached in the particular study. “Transferability” refers to the degree to which the results of the qualitative research can be applied to other contexts or settings.

Sustaining the “trustworthiness” of a research study depends on the issues,

quantitatively, considered as validity and reliability (Lincoln & Guba 1985:266). The idea of discovering truth through reliability and validity is replaced by the concept of

trustworthiness. The “credibility” criteria involve establishing that the results of the

qualitative research are believable from the perspective of the participant in the research. “Confirmability” refers to the degree to which the results of the qualitative research could be confirmed or corroborated by others.

Although objectivity and truthfulness are critical to both qualitative and quantitative research, the criteria for evaluating qualitative research differs from quantitative research, however, as the researcher seeks believability based on coherence, insight, instrumental utility and trustworthiness through a process of verification rather than applying traditional validity and reliability measures used for quantitative research (Miller 1992:196). The value of qualitative research lies in the particular description and themes developed in context to the specific site and particularity is characteristic of qualitative research rather than generalisability. Qualitative reliability shows that the research

approach is consistent across different researchers and different projects while qualitative validity indicates that the researcher checks for accuracy using validity strategies to assess the findings. Several reliability procedures are available, including the checking of transcripts for obvious mistakes and ensuring there is not a drift in the definition of codes (Creswell 2009:190-193).

Since there can be no validity without reliability, a demonstration of the validity is sufficient to establish reliability (Lincoln & Guba 1985:316). To ensure internal validity, the following strategies are used in this case study.

 Triangulation is defined as a validity procedure where researchers search for convergence among multiple and different sources of information to form the themes or categories in a research study (Creswell & Miller 2000:126). In addition, triangulation strengthens a study by combining methods using several kinds of data, including using both quantitative and qualitative approaches to test

for validity (Patton 2001:247). The research conducted in this case study includes describing the extent of asset theft (quantitative method) and the preventive approach (qualitative method) to address asset theft implemented by a South African mine, as well as literature reviews on available research pertaining to crime prevention strategies with the aim of recommending the most viable preventative strategy for the participating mine. If themes are established based on converging several sources of data, it can be claimed that the process has contributed to the validity of the study (Creswell 2009:191).

 Member checking is used in qualitative research methods to determine accuracy by referring the basic themes back to the participants for confirmation (Creswell 2009:191). In this case study the themes are established based on converging the perspectives from participants, the field notes of observations done by the researcher during site visits, and literature available on best practices in crime prevention. A representative of the participating mine was made available to validate the interpretations made by the researcher.

 Spending a prolonged period in the field contributes to the validity of a study as the researcher develops an in-depth understanding of the phenomenon being studied (Creswell 2009192). The researcher developed the themes for this case study over a period of five years during which she had access to the participating mining sites and the participants.

 Peer briefing enhances the accuracy of the study and thereby the validity, as this strategy involves the interpretation of the data beyond the researcher (Creswell 2009:192). A social researcher of the Institute for Security Studies served as a peer reviewer to verify the accuracy of the interpretations made by the researcher as well as the correctness of the statistical findings.

 Secondary audits were applied to the research findings of the case study to ensure statistical validity and replicability. The measurement of the quantitative data was correlated and verified for correctness and meticulous record was kept of all information described in the study or used to base findings on.

 Clarification of possible researcher bias was described in section 1.2.1.1. This self-reflection creates an open and honest narrative that contains comments by the researcher about how his or her interpretation of the findings is shaped by background and experience (Creswell 2009:192).