Reliability, Validity and Triangulation

Scholars insist that quality research uses methods that are reliable and have validity. Reliability is defined according to Scott & Morrison (2007: 208) as;

A measure of quality and the term means repeatability or consistency ... A measure is reliable if it provides the same results on two or more separate occasions, when the assumption is made that the object being mastered has not changed ... Whether quantitative or qualitative measures are used, the key to successfully applying a notion of reliability is that the object being measured remains stable.

I, as the researcher, was aware of the fact that respondents in the research knew me personally from when they were learners on the programme. I was the project manager for the service provider providing the tuition programme. Such a scenario could have a negative bearing on the research and could have the respondents uncomfortable and unwilling to be honest with me given my close association to the service provider, schools from which they were learners, programme sponsor and the fear of being

perceived as ungrateful for the opportunity afforded to them. Acknowledging that I have worked, socialised and had a professional working relationship with this group of

learners, made me much more aware of the need to adhere strictly to research

principles such as reliability, validity and triangulation to ensure that the findings of the research were both authentic and reliable.

I was also aware of problems of subjectivity in an interview setting and tried as a researcher to be as rigorous and objective as possible. Whenever participants’ views were in direct contradiction to programme documentation, I attempted to probe their views to get the fuller picture, hoping to maximise reliability through the probing of

their responses (Scott & Morrison, 2007: 208). Reliability in the research was also influenced by the fact that much of my views were supplemented and given greater depth by the fact that l was an employee of the service provider in question as project manager on the programme for a period of two years, not always as a researcher but also as a member of staff.

Validity, according to Scott & Morrison (2007: 253), has two main aspects;

Internal and external, where the former refers to the accuracy or authenticity of the description being made, and the latter refers to its application to other cases, across place and time. Internal validity is therefore a measure of accuracy and whether it matches reality; external validity, on the other hand, is a measure of generalizability.

Internal validity was strengthened by the fact that I as the researcher knew the

programme and was familiar with many of the issues on the programme. This position allowed me to probe participants more pointedly when responses were ambiguous and not comprehensive (Scott & Morrison, 2007: 253). External validity was ensured, as mentioned earlier by learner participants being almost a representative spread of the type of passes obtained by learners on the programme (except the one learner who obtained an incomplete result and was unavailable for the study).

Triangulation of data consists of “qualitative cross-validation among multiple data sources” (McMillan & Schumacher, 2010: 491). I was aware of the need to triangulate the data collected by using different sources of data collection (document analysis and interviews of various stakeholders of the programmes) to gain better insights into the programme from different perspectives and to corroborate the findings through probing and verification (Scott & Morrison, 2007: 251; 252). Although the views of the funder, service provider and trainers could not be obtained, l relied on programme documents to account for their experiences of the programme.

The service provider employees expressed they felt it risky to take part in the research because of the many tensions in the programme. There seemed to be, as recorded in numerous programme meetings, a lack of trust among all relevant stakeholders

the service provider. Minutes indicate that district officials feared unionised teachers, the programme sponsor feared lack of co-operation from the district and the service provider in turn also feared antagonising all of the relevant stakeholders.

The trainers’ unease to participate could do with their fear of jeopardising the

relationship with their employer or fear of losing their teaching or district official jobs. Some trainers working for the service provider were moonlighting, something that their work contracts discouraged or prohibited. Another reason for the refusal of programme trainers to take part in the study could have been the existence of a non-disclosure agreement signed between them and the service provider, apprehensive that this too could affect their working relationship with their employer.

Cognisant of the fact that this problem of lack of multiple sources of data could

compromise and weaken the research study, I as the researcher tried to rely on more programme documentation (circulars, brochures, press releases and reports) that would provide information about the aim, structure and delivery of the programme. These documents include funder reports, trainer feedback forms, letters from the service provider to parents and caregivers of learners on the programme, service provider information booklets and newspaper sources. For example, interview questions on participants’ experience of the curriculum were compared with tutorial forms completed by them in 2010.