Springer (2010) has argued that both validity and reliability are relevant to the design and interpretation of qualitative studies and both are as desirable in a qualitative study as they are in a quantitative research. According to Ary and Jacob (1990, p. 256), “the validity question is concerned with the extent to which an instrument measures what one thinks it is measuring and the reliability of a measuring instrument is the degree of consistency with which it measures whatever it is measuring”. Creswell (2005, p. 292) define threats as the “problems that threaten our ability to draw correct cause and effect inferences that arise because of the experimental procedures or the experiences of participants”.
The validity of quantitative results refers to the extent with which the statistical results reflects the real meaning of constructs that are under observation. Qualitative validity means that the researcher checks for the accuracy of the findings by employing certain procedures. This validity refers to the congruence between the explanations of the phenomenon and the realities of the world (Creswell, 2003; McMillan & Schumacher, 2010). The differences between quantitative and qualitative views of validity that are highlighted by Babbie and Mouton (2008) and are depicted in Table 1.
Internal and external validity in terms of quantitative data refer to whether the questions asked are actually asking what the researcher wants to know. The reliability of quantitative
results can be gauged according to whether a test returns the same results repeatedly. The reliability of the results of a study is revealed by statistical treatment of the numerical survey results (Cronbach α scores). Quantitative measurements will have good test-retest reliability if
repeated administration of measurement yields the same results (Creswell, 2003; Springer, 2010).
Table 1
A comparison of notions of validity in quantitative and qualitative research (Babbie & Mouton, 2008)
Quantitative Qualitative
Internal validity Credibility
External validity Transferability
Reliability Dependability
Objectivity Confirmability
In terms of qualitative research validity, truthvalue, or enhancing the credibility of the research, is attained by the triangulation of data and checking the findings against the literature (Leedy & Ormrod, 2001). Applicability refers to how transferable the findings are to other contexts or settings (Krefting, 1991). For this aspect broad sampling is required to achieve a degree of applicability; that is, the participants should be carefully selected to reflect the diversity of people to achieve what may be considered a qualitative (not quantitative) representative sample of the population from which to draw general conclusions. Qualitative generalization is a term that is only used in a limited way in qualitative research, since the intent of this form of inquiry is not to generalize findings to individuals, sites, or places outside of those under study. The value of qualitative research lies in the particular description and themes developed in context of a specific site. Particularity rather than generalizability (Creswell, 2003)
is the hallmark of qualitative research. However, Yin, (2003) argues that qualitative case study results can be generalized to some broader theory. The generalization occurs when qualitative researchers study additional cases and generalize findings to new cases (Creswell, 2009).
Transferability relies on the data and comparisons with other research in the literature. Consistency refers to the question of how much of the findings could be replicated if the study were repeated in the same context and with the same participants (Lincoln & Guba, 1985). Dependability is the criterion used for determining consistency and is usually achieved by the triangulation of data and the interpretations made (Struwig & Stead, 2001). Neutrality refers to the extent to which the research findings and the perspectives of the researcher appear to be free from bias and is determined by the criterion of confirmability (Krefting, 1991). Steps taken to increase the degree of confirmability are comparing findings to other research literature and post- interview discussions between researcher and observer to check for consensus on perceptions (Groenewald, 2004).
All of the above interpretations of validity were kept in mind when undertaking this study, as well as the ethical considerations required in order to maintain the human dignity of the participants in the research process (Cohen, et al., 2000).
5. ETHICAL CONSIDERATIONS
As noted above, ethical issues are concerned with human dignity during research. Hopf (2004, p. 334) states that “Under the keyword „research ethics‟ it is usual in social sciences to
group together all those ethical principles and rules in which it is determined – in a more or less binding and more or less consensual way – how the relationships between researchers on the one hand and those involved in sociological research on the other hand are to be handled”. As
researchers anticipate data collection, they need to respect the participants and the sites for research because many ethical issues arise during the process (Creswell, 2009). Because the subjects of enquiry in interviewing are human beings, researchers must take extreme care to avoid any harm to them (Denzin & Lincoln, 2003). The researcher thus needs to consider the special needs of vulnerable populations, such as minors, mentally incompetent participants, victims, persons with neurological impairments, pregnant women or foetuses, prisoners and individuals with HIV. Researchers need to file research proposals containing the procedures and information about the participants to the Ethic Committee so that they can review the extent to which the research being proposed subject individuals to risk (Creswell, 2009).
When the researcher analyses and interprets qualitative data, issues emerge that call for good ethical decisions. In anticipating a research study the researcher must consider the following:
How will the study protect the anonymity of individuals, roles, and incidents in the project? In qualitative research for example, inquirers use aliases, or pseudonyms for individuals and places to protect identities
Data, once analysed, need to be kept for a reasonable period of time (5-10 years). Investigators should then discard the data so that it does not fall into the hands of other researchers who might misappropriate it.
The question of who owns the data once it is collected and analysed, can be an issue that splits research teams and divides individuals against each other. A proposal might mention this issue of ownership and mention how it will be resolved, such as through the development of a clear understanding, between the researcher, the participants and possibly the faculty advisors.
In the interpretation of data, researchers need to provide an accurate account of the information. In qualitative research, this accuracy may include using one or more of the strategies to check the accuracy of the data with participants or across different data sources. (Creswell:2009)
Ethical principles may include the following:
Approval from an Ethics Committee (Zuber-Skerrit, 1996; Denscombe, 2003) and the related departments
Principle of informed consent (Hopf, 2004, Heaton, 2004; Mason, 2002) and voluntary participation (Henn et al., 2006)
Termination of participation at any point in time (Henn et al., 2006)
Truthful reporting without harming the participants or betraying their confidentiality (Creswell, 2003; Hopf, 2004), results for the “good of society” (Henn, et al., 2006, p. 71)
Secure and safe keeping of data (Denscombe, 2003; Heaton, 2004) Restrict access and distribution of data (Denscombe, 2003)
Anonymity during reporting (Cohen, et al., 2000) and using and reporting of the data as originally specified (Denscombe, 2003)
Ethical issues do not stop with data collection and analysis; issues apply as well to the actual writing and dissemination of the final research report. For example:
Reports should not use language or words that are biased to persons because of gender, sexual orientation, racial or ethnic group, disability or age.
Issues including the potential suppressing, falsifying or inventing findings to meet a researcher‟s or audience‟s needs need to be considered and counteracted.
In planning a study it is important to anticipate the repercussions of conducting a research on certain audiences and not to misuse the results to the advantage of a group or another. The researcher needs to provide those at the research site with a preliminary copy of any publications from the research.
An important issue in writing a scholarly manuscript is to not exploit the labour of and to provide authorship to individuals who substantially contribute to publications.
It is also important to release the details of the research with the study design so that readers can determine for themselves the credibility of the study (Creswell, 2009).
In this study informed consent was requested from participants after prior permission to conduct this research as part of the DG Murray project offered by the Centre for Educational Research, Technology and Innovation at the NMMU. After obtaining ethics clearance, the researcher approached Department of Education officials, principals and teachers of the participating schools. Their roles as participants, right to choose to be participants and to participate or not in this study were explained to them. They were assured of confidentiality that participation was voluntary and were given a guarantee that they could withdraw from the study at any time and that no personal details would be disclosed. Confidentiality of the information collected in the schools was also ensured and that no portion of the data collected would be used for any purpose other than this research.
6. SUMMARY
This chapter describes the research design and methodology that was followed to address the research problem. The choice of working within a pragmatic paradigmatic framework is explained and the advantages of using both qualitative and quantitative data are motivated. The data collecting methods that were used, i.e. questionnaires (both pencil-and-paper and online) and interviews are also described. The method of data analysis is explained and limitations and weaknesses in the design are made explicit. Issues of validity, reliability, and the generalizability of the research findings are discussed and ethical issues are considered.
CHAPTER FOUR
RESULTS - FIRST STUDY
1. INTRODUCTION
This chapter reports on data generated in this first study using the methodology and research instruments that have been described in detail in the previous chapter (Chapter three). The instruments consist of open-ended questionnaires and interviews about challenges faced by principals in historically disadvantaged schools in the implementation of curriculum policies in terms of their role as instructional leaders.
As indicated in chapter three, this study has used both qualitative and quantitative approaches to add value and diverse perspectives. Data were generated from questionnaires administered to principals and teachers in selected schools in the Uitenhage and Port Elizabeth districts as well as Department of Education (DoE) officials. Data were also generated from interviews conducted with selected principals and DoE officials in these districts. These data are presented in this chapter.
Findings were allocated into themes deductively from those identified in the literature review and new themes were developed inductively from the data using thematic analysis processes. The data obtained from principals, teachers and DoE officials are presented in the following sequence as:
Principals‟ responses Teachers‟ responses and
Each of the above categories of participants was given a different set of questions to determine their respective perceptions regarding the principal‟s role in curriculum
implementation and instructional leadership. The first part of the questionnaires required that each participant indicate their designation, age in years, gender, formal qualifications and teaching experience to develop a profile of leadership in schools. These data are presented in the demographics section below.