Validity and reliability

In this study content, construct and face validity were used (Bless et al. 2006:135; Cohen et al., 2005:105). Validity is generally associated with whether the research instrument measures what it intended. Bless et al. (2006:135) maintain that unless we are sure our techniques actually measure what is intended, we cannot be certain of what the results mean.

However Cohen et al. (2005:105) state that in recent understandings of the concept of validity, it “can be addressed through the honesty, depth, richness and scope of the data achieved, the participants approached, the extent of triangulation and the disinterestedness or objectivity of the researcher”. The most important types of validity are noted as content validity, criterion-related validity, construct validity, and face validity.

3.8.1 Content validity

In this study the researcher sought to investigate both technical and non-technical factors, and the sub-parts of these two broad categories that influence e-Learning practice. In social research, phenomena that are studied are complex, and comprise multiple parts. Content validity requires that for an understanding of the whole, all of these parts need to be understood. This approach is supported by Cohen et al. (2005) and Bless et al. (2006) as follows.

Cohen et al. (2005:109) maintain that an “instrument must show that it fairly and comprehensively covers the domain or items that it purports to cover”. This is similar to Bless et al. (2006:136), who contend that for content validity, a researcher “must find a technique which will provide information on all its different components”. In this study the researcher sought to investigate both technical and non-technical factors, and the sub-parts of these two broad categories that influence e-Learning practice.

3.8.2 Construct validity

Theories of technology adoption and diffusion, motivation, self-efficacy and expectancy were used in this study to enable construct validity. The researcher wanted to establish a link between the measurement technique to be used and known theories in the field of study, and felt that if the link were close, then a higher level of validity could be attained (Bless et al. 2006:138). Cohen et al. (2005:110) argue that to maintain construct validity, a researcher’s construct of a particular issue must be in agreement with other constructs of the same issue. This they believe can be achieved through parallels with other forms of measurement of the phenomena.

In this study the researcher sought to investigate e-Learning practice in terms of the understandings of the constructs of integration, adoption, ease of use, usefulness and benefits.

3.8.3 Face validity

The interviews in this study sought to elicit opinions and beliefs about the use of technologies for e-Learning. It was thus important to ensure that the instrument used would be valid and reliable. Face validity is aligned to the overall notion of validity, that is, the plausibility of an instrument to do what it is intended to do. Cohen et al. (2005:132) note this as “that, superficially, the test appears to test what it is designed to test”. The notion of “at face value” (Cohen et al., 2005:132) is echoed by Babbie and Mouton (2001:642), who state face validity as “that quality of an indicator that makes it seem a reasonable measure of some variable”. The researcher has after careful design, and checking and testing of the instrument, reached the conclusion that the interview leads and questions were able to elicit the required data.

In ensuring validity in this study, an online survey, literature reviews and semi- structured interviews were the three main sources of data collection. Each of the three sources produced data which could be analysed either qualitatively and/or quantitatively. The use of many data sources to collect evidence of the same phenomenon is indicative of triangulation which increases the validity of a study (Cohen et al., 2005:112-113). In this study, methodological triangulation was used in that both qualitative and quantitative methods were employed. The quantitative analysis was used at a basic level to support the qualitative data where relevant, and to look for emerging trends.

The researcher needed to ensure that the instruments could be used with the different participants in their own context and yield similar outputs. If this could be done using the same methodology with similar outputs, then the instrument and methodology of application could indicate reliability. The notion of reliability is summed up as focusing on the consistency of measurement, that is, consistent and

replicable “over time, over instruments and over groups of respondents”; in other words, whether the same results will be achieved in different contexts (Bless et al. 2006:130; Cohen et al., 2005:117).

The researcher noted that to provide trustworthiness and legitimacy to the study, the criteria in deciding which forms of data analysis to undertake would need to be governed by their fitness for purpose (Cohen et al., 2005:82). The data was subjected to cluster analysis using both inductive and deductive techniques. Using this approach and technique symbiotically promoted insight into the phenomena of e- Learning practices.

The conventional views of reliability and validity, according to Cohen et al. (2005:129), in referring to the work of Lincoln and Guba (1985), is equated to the concept of ‘trustworthiness’. Four concepts are associated with the notion of trustworthiness.

These are: credibility, confirmability, transferability, and dependability. A similarity in understanding of trustworthiness is noted by Maxwell (1992:285) in the identification of categories of validity: descriptive, interpretive (similar to credibility and confirmibility), theoretical, generalisable (similar to transferability) and evaluative validity (similar to dependability).

The research approach to addressing credibility and confirmability was to make available the instruments and all data for scrutiny, comparison and confirmation by other researchers. This included the raw data from the survey, the leads and questions from the semi-structured interviews, the transcribed interviews, and the reference list for the literature analysis.

The analysis and discussion of the data included direct quotations from transcriptions of the interviews and from the open-ended questions in the survey. These were subjected to member checking for verification.

In this study, dependability is linked to credibility and confirmability. The reference list, and where available reference web links, are made available so that authenticity

of the study and findings can be checked. Direct transcriptions of the audio-recorded interviews and exact raw data from the survey are available. These constitute an audit trail of evidence collected during the study.