Data collection Instruments

In the quantitative phase of this study, to answer the data collection instruments that were used research questions 1, 2 and 3 were Amharic version of Felder-Soloman’s Index of learning styles (ILS), and a multiple choice chemistry test/tasks. The Amharic version of Felder-Soloman’s Index of Learning Styles (ILS) was used to identify students’ learning styles. Whereas their academic performance on some fundamental concepts in chemistry were measured using multiple test. Because, according to Burton, Sudweeks, Merrill, and Wood (1991) multiple choice test is appropriate instruments to measure different educational objectives and apply in many subject areas. It is also easy and efficient to administer. Andrews (2012) stated that the reliability and validity are essential qualities of a test for evaluation. Therefore, Andrews (2012) reported that the coefficient of internal consistency reliability of multiple choice test is higher than for essay.

The multiple choice items of the chemistry tests/tasks are designed from selected areas of chemistry topics in each grade level chemistry syllabus which were covered in the first semester. The chemistry topics were selected based on fundamental concepts in chemistry identified from literature on the philosophy of chemistry (Appendix A) and based on the emphasis given in the curriculum, and the extent of difficulty of the topics as pointed by experienced school teachers. It was literatures on the philosophy of chemistry that helped the researcher to identify some fundamental concepts in chemistry that could form the logical structure of the discipline.

In the qualitative phase of the study, to answer research questions 4 and 5 a qualitative data were collected using Amharic version of a semi-structured and open-ended questionnaire along with oral explanations of each questions on a face to face basis.

71

The reliability and construct validity of the ILS

According to Felder and Spurlin (2005), the test-retest reliability research reveals that ILS has high and statistical significant correlation. Moreover, Livesay et al. in Zywno (2003) found that ILS has relatively high test-retest reliability in repeated measurements over time, and concluded that the ILS was an appropriate and statistically acceptable tool for characterizing learning preferences.

Different authors reported that, the English version of Felder and Soloman Index of Learning Style (ILS) has an acceptable level of construct validity and inter-consistency reliability for research purposes (Felder & Spurlin, 2005; Ku & Shen, 2009; Litzinger, et al., 2007; Litzinger, Lee, Wise, & Felder, 2005; Zywno, 2003). Moreover, Zywno (2003) states that if cut-off value for Cronbach’s alpha (α) is greater than 0.5, it is acceptable for attitude tests.

The level of difficulty of the English version of the ILS questionnaire was evaluated by 12 students, whose academic record in the school ranges from low to high and then translated into Amharic. These students were not participants of the study. Most of the students reported that the languages of some ILS items were difficult. And they suggested that it should be translated into Amharic language, in which they were fluent in writing and speaking through Amharic.

Consequently, the English version of ILS was translated into Amharic language by retaining its psychometric characteristics through triangulation of the translations. In other words, the ILS instrument had been translated into Amharic by one documentary linguistic and culture PhD student, and one psychology lecturer. Following the translation I compared the two Amharic versions of ILS were compared and finally the Amharic version of ILS was produced. And then

72

the researcher (I) arranged discussion for 20 minutes with both the PhD student and psychology lecturer to refine it well. At last the final Amharic version of ILS was given to English lecturer to translate it back into English. Then the two English versions of ILS were compared and found consistent.

In the end, the final Amharic version of ILS was piloted on 25 students and the reliability coefficients (Cronbach alpha) were computed. The pilot study revealed that a Cronbach alpha coefficient of Visual/Verbal was 0.73, Sensing/Intuitive was 0.73, Active/Reflective was 0.68 and Sequential/Global was 0.64. These reliability coefficients show that for each dimension, the ILS was greater than the cut-off value 0.5. Thus, it has acceptable level of reliability for research use. Moreover, the comparison of reliability report (alpha values) for the Amharic version and English version of ILS was nearly the same (see Table 1 below).

Table 4.1 Comparison of the alpha values of the English version and the translated Amharic version of ILS

Acti-Refle. Visual- Verb Sens-Intui Seq-Glob N Source

0.60 0.74 0.77 0.56 572 (Litzinger, et al., 2005)

0.61 0.76 0.77 0.55 448 (Litzinger, et al., 2007)

0.68 0.73 0.73 0.64 24 (the current pilot study)

The table shows that the alpha values for ILS to measure each dimension of Felder-Silverman learning styles were similar and were above 0.5.

73

Reliability and validity of the multiple item chemistry tests

Chemistry tests for measuring academic performance on some fundamental concepts in chemistry were carefully constructed from the fundamental concepts of chemistry in grade 11 and 12 chemistry syllabus. According to Caldin (2002) and Schummer (2003, 2006) some fundamental concepts in chemistry are: pure substance, chemical species, compound, affinity, chemical reaction, atom and subatomic particles, molecules and molecular structure, practical method (experimentation), energy, chemical theories, and cognitive method of chemistry (pictorial language of chemistry, model building & representation).

Therefore, to construct the test, the researcher identified some fundamental chemical concepts from the topics included in the first semester of grade 11 chemistry syllabus and in the first semester of grade 12 chemistry syllabus (see Appendix A). The chemistry test for measuring academic performance of grade 11 students was constructed carefully from fundamental chemical concepts in the topics: Atomic structure and periodic table, and chemical bonding and structure in grade 11 chemistry syllabus (see Appendix A: table 1 & 2). Moreover, the chemistry test for measuring academic performance of grade 12 students was constructed carefully from

fundamental chemical concepts in the topics: Acid-base equilibrium and common thermodynamic terms in grade 12 chemistry syllabus (see Appendix A: table 1 & 2).

To ensure content validity of the test items the relative proportion or number of items constructed from each topic was calculated based on the weight given to each of them, and their relative weigh were determined by proportions of periods allotted to each topic (Gronlund, 1977). That is:

74

After the test was constructed it was validated by two experienced chemistry teachers who were teaching in both grade levels in preparatory school “A” and one experienced chemistry teacher who was teaching in both grade levels in preparatory school “B” and my supervisor (chemistry educator).

Before administration, each test was piloted on 25 students in other preparatory schools at their respective grade levels and KR-20 was computed. The computation of KR-20 reliability test showed that the reliability index for the 21-item chemistry test used to measure grade 11 students’ academic performance on some fundamental concepts in chemistry was .90. Similarly, the computation of KR-20 reliability test showed that the reliability index for the 22-item chemistry test used to measure grade 12 students’ academic performance on some fundamental concepts in chemistry was .87. Therefore; this indicates that the tests were reliable enough to measure students’ academic performance on some fundamental concepts in the aforementioned chemistry topics.

Semi-structured and Open-ended Questionnaire

To examine the role of chemistry instructional materials on students’ academic performance, the researcher designed a semi-structured and an open-ended questionnaire accompanied by oral explanations (see appendix B). The language clarity of the questionnaire was evaluated by one Amharic lecturer and two preparatory school students. And then the researcher sat together with the lecturer and students, and discussed on the clarity of the questionnaire. Moreover, while participants were filling the questionnaire the researcher was with the participants if further explanations were needed by them. In other words, the questionnaire was designed to be filled on

75

a face to face basis in the office of chemistry department for giving explanations to participants when the need arises.