The quality of research is described by validity and reliability of data. This study involved use of both qualitative and quantitative data. Guba & Lincoln (1981) stated that the nature of knowledge within the quantitative paradigm is different from the knowledge in qualitative paradigm. They noted that, within the quantitative paradigm, the criteria of quality are internal validity, external validity, reliability, and objectivity. On the other hand, they proposed four criteria for judging the soundness of qualitative research to ensure "trustworthiness" and credibility. These criteria were; credibility, transferability, dependability, and
72
confirmability (Guba & Lincoln, 1981; Trochim, 2006). Validity and reliability of data both quantitative and qualitative data was therefore considered.
3.7.1. Validity and reliability of quantitative data
For the empirical problem analysis, questionnaires were developed on the basis of responses received from two experts from the university who evaluated them for content validity before they were used. Piloting was done to determine the clarity of question items in the questionnaires. This was to enable the researcher to check the suitability of the instruments to collect required data. Piloting was done in three schools. Questions that elicited ambiguous responses were adjusted accordingly. After piloting, the respondents were allocated random numbers. Using split-half method each type of instrument was split into two groups, those from even and those from odd numbered respondents. The reliability of the two was correlated by the Spearman-Brown split-half coefficient:
α = 2rxy / (1+rxy)
Where: rxy is the reliability coefficient between two variables x (odd) and y (even). The reliability coefficient rxy is determined by Pearson product- moment correlation formula; rxy= Σxy- (Σx) (Σy) __ n_____ ___________________ √ [ Σx- (Σx)2] [Σy- (Σy)2] n n
73
Σ x represents sum of scores of respondents of one (odd numbered) group, Σy represents sum of scores by the second (even numbered) group of respondents and n represents the number of respondents for each group.
For teachers‘ questionnaire the reliability of the Likert type items was found to be 0.814. These values are above the value 0.70 cited by Siegle (2010) as minimum reliability required for research purposes. The instruments were therefore considered to give reliable data.
3.7.2. Validity and reliability of qualitative data
For the other part of the study (stages 2-4), qualitative research methods were employed. The proposed criteria for judging the "trustworthiness" of qualitative research is credibility, transferability, dependability, and confirmability (Guba & Lincoln, 1981; Morse et al, 2002; Trochim, 2006).
According to Trochim, (2006) the credibility criterion involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in the research. This was achieved through formative evaluation of the lessons as carried out in the classrooms and responses from the teachers and the learners. Transferability refers to the degree to which the results of qualitative research can be generalized or transferred to other contexts or settings (Trochim, 2006). The researcher enhanced transferability through a thorough description of the research context and the assumptions that were central to the research.
74
Dependability emphasizes the need describing the changes that occur in the setting and how these changes affected the way the researcher approached the study (Trochim, 2006). This can be achieved by various evaluation activities in the study being embedded in a cyclic approach of design and formative evaluation for the development of instructional materials (Ottevanger, 2013). The design of this study was iterative in nature with results of one stage informing the next stage. Confirmability refers to the degree to which the results could be confirmed or corroborated by others (Trochim, 2006). It is a measure of objectivity in qualitative research. In the light of confirmability, the researcher has documented the procedures for checking the data throughout the study. After the study, one can conduct a data audit that examines the data collection and analysis procedures and makes judgments about the potential for bias or distortion (Trochim, 2006).
Morse, et al (2002) also argues that the standards for the post hoc evaluation of qualitative inquiry (which can be cited at the end of a study) is to determine the extent to which the reviewers have confidence in the researcher‘s competence in conducting research following established norms but cannot be used as verification strategy. The verification strategies that can be used by the researcher in the process of inquiry so that reliability and validity are actively attained include; methodological coherence, appropriate sampling, iterative interaction between data collection and analysis, thinking theoretically, and theory development (Morse et al, 2002)
75
This study used Design Based Research design that uses mixed methodology (DBRC 2003) which helps the researcher to ensure: (a) ability to discover new perspectives (b) ability to examine complementary facets of a phenomenon, (c) triangulation (d) development and iteration (using methods sequentially, so that results from the first method inform the second method), and (e) expansion (Aksela, 2005). The characteristics of design research used in this study enhanced the ability to attain the needs of trustworthiness regarding methodological coherencies.
Validity of findings in design research is addressed by the partnerships and iteration, which results in increasing the alignment of theory, design, practice, and measurement over time (DBRC, 2003). This study was iterative in nature with the outcome of one process informing refinement of the materials for the next stage and repetition of the processes. The users of the materials evaluated them and suggested refinements that would make the materials better. Research results that are validated through the consequences of their use provide consequential evidence or validity (Barab & Squire, 2004). Reliability of design research findings and measures can be promoted through incorporating multiple data sources (DBRC, 2003). Multiple data sources were used throughout the study process.
Investigator responsiveness determines the reliability and validity of the study (Morse, et al 2002). In this study, the researcher carried out lesson observations and allowed for flexibility in the use of lesson materials. The researcher was able to
76
note important features emerging from classroom activities that were not indicated in the observation guide and used them to refine the materials.
Ottevanger (2013) incorporated quality criteria based on relevance, consistency, practicality and effectiveness. In this study, relevance was attained through review of literature, content analysis of instructional materials, lesson observation to identify needs of investigative practical work, generation of design specifications for the materials and expert appraisal by chemistry teachers. Consistency was attained through expert appraisal and formative evaluation of prototypes. Practicality was determined by users during the try-out as well as during the final evaluation and user appraisals while effectiveness was assessed during classroom use of the materals.
According to Nieveen (1999) the reliability and validity of the instructional materials can be based on the three aspects of quality of exemplary materials suggested which are; validity, practicality and effectiveness. These basically formed the criteria against which the designed instructional materials were evaluated.