Validity and reliability

Quality and validity are highly important in any study; “research needs to be defensible to the research and practice communities for whom the research is produced and used” (Onwuegbuzie & Johnson, 2006, p. 48). Concerns about validity and reliability can be allayed by paying careful attention to how the data collection, analysis, and interpretation process is carried out, and how the results are presented (Merriam & Tisdell, 2016). Both qualitative methods and quantitative methods have their own criteria for determining the quality and validity (Creswell & Plano Clark, 2011; Maxwell & Mittapalli, 2010). However, the issue of measuring or determining the validity of mixed methods research approaches is more controversial and researchers often disagree regarding the measures to use to determine the validity or quality of a mixed method study (Bryman, 2006; Onwuegbuzie & Johnson, 2006; Tashakkori & Teddlie, 2003). Nevertheless, most researchers agree that the mixed method approach per se is neither more nor less valid than either the qualitative or the quantitative approach (Bazeley, 2004).

Triangulation, often used in mixed method approaches that use a variety of data collection techniques. According to Merriam and Tisdell (2016), triangulation is a powerful tool that can be used to increase the internal validity of research that uses multiple methods of data collection and analysis. A mixed methods approach increases validity by using the quantitative data to back up the qualitative data and vice versa, linking and cross-checking between the two methods. Labelled “inter-triangulation” by McMurray, Pace, and Scott (2004), the simpler term “triangulation” is also used in the literature. Inter-triangulation allows the researcher to discover a more precise perception of reality (to approach the topic from a subjectivist paradigm) or to discover the true nature of a phenomenon’s reality (to use the explanation of

83

the objectivist paradigm). Because the questionnaire was anonymous, this study was not able to triangulate the interview data with individual interviewees’ responses to the questionnaire. However, the interviews were still able to provide more detail and highlighted some interesting aspects of the phenomenon of preparedness. As pointed out by Firestone (1987), qualitative data give a reader enough detail to show that the conclusions are intuitive and congruent with reality.

As described in Section 4.5, considerable effort was taken to ensure that an appropriate and large enough sample for the survey (and interviews) was chosen to ensure that the results are representative of the wider female mathematics teacher cohort in Saudi Arabia. To represent mathematics PSTs’ perceptions of preparedness to teach across the wider population, I took my sample from as many education institutions as possible around Saudi Arabia.

To maximise the response rate and to ensure that the survey would yield high-quality data, the survey design paid careful attention to the order in which questions were presented, the wording of the questions (adapted from pilot study (Section 4.4.3), the layout and the instructions accompanying the questionnaire (Cohen et al., 2011). In order to make the questionnaire easy to complete without inducing fatigue, a Likert-type scale was used (Ben-Nun, 2008). Superficial responses are undesirable, as they reduce the validity and reliability of the data being collected. Following Krosnick’s (1999) suggestion, participants were told that their responses were valuable and very important to the researcher, which is also believed to keep satisficing to a minimum.

According to Merriam and Tisdell (2016), following sound ethical practices help to establish the trustworthiness of a study, as ethical protocols encourage respondents to answer honestly, fully and without bias. It was noted that the pilot questionnaire resulted in extremely positive responses beyond what would intuitively be expected. This was potentially a consequence of Saudi culture, where students, and especially female students, are discouraged from disagreeing with or giving negative responses to authority figures; possibly, my role as a researcher put me into the category of an authority figure. My reassurance that their responses would be kept confidential and their anonymity would be preserved was designed to help overcome this reticence. When starting the main phase of data collection, I also reminded the participants to read the information sheet carefully and this was followed by a request to answer the questions honestly and credibly. As a result, PSTs in the main study provided more varied range of answers, and some went into great detail in their answers to the open-ended questions.

84

It is also possible that another cultural feature may have been at work here. It has been noticed in other studies that in non-Western contexts, PSTs may be reluctant or reticent about reporting a low level of preparedness or similar negative self-reports (Lin & Gorrell 2001; Lin, Gorrell, & Taylor, 2002; Yeung & Watkins, 2000).

As a final step to help ensure the validity of the study, I aimed to be methodical when presenting the results of my research in order for the findings to have “sufficient clarity (e.g., in separate texts, tables, and exhibits) to allow readers to judge independently later interpretation of the data” (Yin, 2012, pp. 14 – 15).