STAGE 10: Validity and Reliability

At this stage of the study, the research had gone through all of the processes needed in analysing data collected and results presented. It is however important to note, that, for this research investigation to be meaningful, the outcome or result of the study must be valid. If the outcome does not measure what the researcher intends to investigate then the results cannot be used to address the main aim of this study, the research question (Collis & Hussey, 2014). It cannot also be used to generalize any findings and becomes a waste of time and effort. In order to ascertain how appropriate, useful and meaningful a research is, it is very important to ensure its validity and reliability (Blaike, 2014).

5.13.1 Research’s Reliability

Joppe (2000, p. 1) defines reliability as: ‘The extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable.’ Collis & Hussey (2014, p. 52) also define reliability as ‘the accuracy and precision of the measurement and absence of differences in the results if the research is repeated’.

A common feature to the definitions above is the idea of same results even when repeated irrespective of whoever is conducting it. However, there are three key points according to Bryman & Bell (2015, p. 48), which is widely accepted that characterizes reliability in any quantitative research study. These are: (1) the degree to which a measurement, given repeatedly, remains the same (2) the stability of a measurement over time; and (3) the similarity of measurements within a given time period.

5.13.2 Research’s Validity

The traditional criteria for validity are traced to the positivist practice. By implication, it could mean that positivism is to an extent defined as a systematic theory of validity. To the positivists, validity was a thing inherent and was as the result of culmination of other empirical conceptions: universal laws, evidence, objectivity, and mathematical data to name just a few (Winter, 2000). Joppe (2000) describes in the following explanation what validity is in quantitative research:

‘Validity determines whether the research truly measures that which it was intended to measure or how truthful the research results are. In other words, does the research instrument allow you to hit "the bull’s eye" of your research object? Researchers generally determine validity by asking a series of questions and will often look for the answers in the research of other’ (p. 1)

If the issues regarding reliability, validity, trustworthiness, quality and rigor are meant to differentiate a 'good' research from a 'bad' one, then testing and enhancing them in a research investigation will be of much importance in any of the paradigms.

5.13.3 Reliability and validity of this research

There are numerous ways to check for validity and reliability of a given research paper as discussed above but the researcher tested them using the main methods of data collection (questionnaire and interviews) used for this study.

5.13.3.1Questionnaires

The process of designing questionnaires for this study is summed up in the figure 5.5 above showing the type of question, their wording and the order in which they are presented. This section explores how the validity and reliability of the questionnaire was tested. First and foremost, the researcher in an introduction letter (or covering letter) explained the rationale behind this study to clarify the context in which these questions are being asked. This was done in simple language to easily convey the actual meaning of this research project to the layman (respondents) in return for an honest response. Contents were taken from literature/theoretical frameworks to generate statements/questions for the questionnaire. A link was then established between the objectives of the study and their translation into content.

5.13.3.2 Pilot testing

Pilot test was conducted at the starting phase of the data collection process. Pilot study constitutes the mini versions of the full-scale research study. In other words is takes into consideration the specific pre-testing of particular research instrument, which in this case is the questionnaire and interview schedule. This is also called the feasibility studies (Creswell 2007; Lincoln & Guba, 1985).

It is that which assisted the researcher to refine the questions, evaluate its validity, ensure data collected matches research questions and assist in developing the final version of the survey (Collis & Hussey, 2014). One of the advantages of conducting a pilot study is that it gave the researcher advance warning about where the main research project could fail, where research protocols may not be followed, or whether proposed methods or instruments are inappropriate or too complicated. In the words of De Vaus (1993: 54), “Do not take the risk; Pilot test first”. The wording and the order of the questions, the range of answers on multiple-choice questions were also piloted. In addition, the different ways of distributing and collecting the questionnaires were also tested. Results of the pilot study have been fully discussed in chapter 6 of this thesis.

5.13.3.3 Member checking

Furtherance to ensuring the validity of this study, the researcher employed member checking as second technique. Member checking is primarily used in qualitative inquiry methodology and is defined as a quality control process by which a researcher seeks to improve the accuracy, credibility and validity of what has been recorded during a research interview (Barbour, 2001; Byrne, 2001; Coffey & Atkinson, 1996; Doyle, 2007; Lincoln & Guba, 1985). Member checking is also known as participant verification (Rager, 2005), informant feedback, respondent validation, applicability, external validity, and fittingness (Morse et al., 2002).

First and foremost, after the questionnaire or interview questions were designed, the researcher conducted pre-test. Under the pre-test, the questionnaires were given to experts in the field or researchers working in similar research domain to critically evaluate each question. This provided great assistance to the researcher in improving the questionnaire where necessary. The questionnaires were also sent to the research supervisor and other academic supervisor from the University of Salford to confirm the validity of each question.

Another kind of member checking for this study was conducted during the interview process. The researcher transcribed and summarized the information given by the respondents and then questioned the respondents to determine accuracy. During this process, some participant disagreed that the summary does not reflect his/her views,

feelings and experiences, amendment were then made. However, some participants also affirmed accuracy and completeness, leading to the credibility of the study.

Whether the member checking occurs simultaneously during the interview or near the end of the project, these member checks are not without fault. However, they serve to decrease the incidence of incorrect data and the incorrect interpretation of data, with the overall goal of providing findings that are authentic and original (Creswell, 2007; Moustakas, 1994). The greatest benefit of conducting member checks is that it allowed the researcher the opportunity to verify the accuracy and completeness of the findings, which then helped to improve the validity of the study (Cohen & Crabtree, 2006).

5.13.3.4 Peer debriefing

In the field of social sciences research, qualitative researchers seek to understand the world through the perspective of others. This approach yields useful and highly in-depth information regarding people’s motivation, concerns and behavior (Brigham and Joanning, 1999). Peer debriefing requires the researcher to work together with one or several colleagues who hold impartial views of the study. The impartial peers examined the researcher’s final report and general methodology, and feedback was given to enhance the credibility and ensure validity.

With this regards, the researcher attended 2015 UKAIS conferences during the starting phase of the research study (See appendix I for confirmation of attendance), and the SPARC 2016 during the middle stage of the study. Dr. Peekay Richardson, Senior Lecturer at the University of Manchester Business School also validated the findings of the study. This provided close collaboration between the researcher and external reviewer to offer constructive criticisms about the study and offer further and better ideas where necessary.