4.1 Introduction
The purpose of the study was to examine the effects of using a software system called CAT (Computer-Aided Thinking), as compared to traditional software, on students’ achievement as they learned a unit in biochemistry from the UAE 12th
grade
curriculum. CAT provides students with different activities, questions and prompts that match their thinking styles based on the Mental Self-Government Thinking Styles Theory. This chapter presents an overview of how the study was carried out. It outlines the parts that constituted the study, and the phases the study passed through. The chapter contains sections about research hypotheses and questions, participants, treatment of instruments, research design and analysis method.
4.2 Phases of the Study
In order to achieve the purpose of the study, it was organized into two phases. The first phase consists of two tasks:
• Standardize and normalize the Sternberg-Wagner Self-Assessment Inventory of thinking styles for the UAE environment.
• Phase 1 prototype and study.
Chapters 5 and 6 discuss these two tasks in detail.
In the second phase the CAT system was developed and then was used in the main case study to explore the research hypotheses. Chapters 7 to 9 discuss phase 2 in detail.
4.3 Research Hypotheses:
The research hypotheses for the project are:
• Students who use CAT significantly outperform students who use traditional CAI on an achievement test.
• Students who use CAT are more likely to comprehend the knowledge contained in CAT.
• Students who use CAT are more likely to like the subject contained in CAT. • Students who use CAT are more likely to know their preferred way of
learning.
• Students who use CAT are more likely to like the teaching method contained in CAT.
• Students who solve activities in the sequence according to their thinking styles are more likely to understand the knowledge they learn and be motivated.
4.4 Design of the Study
Method
For the study, the effect of the CAT system on students’ achievement was determined using an experimental method. The results were analysed quantitatively and qualitatively.
Subjects
The study went through two phases. In both phases all subjects who participated in the study were male students from Khalifa Secondary School in Abu Dhabi. The same chemistry teacher taught all the students in both phases.
In the first phase, 13 students from a 10th grade class participated. The class was selected randomly.
In the second phase, the subjects were students from two 12th grade classes. There were 30 students in each class. Hence, the sample for the 2nd phase consisted of a total of 60 students. A class was selected randomly from the two as the control group and the other was the experimental group. The students in both the control and experimental groups were of mixed abilities. It is normal practice that at the beginning of the school year, the school administration would allocate students in both classes evenly based on the students' achievement in the previous year. However, during the school year and before starting the experiment, the students of both groups had taken four chemistry assessments on material learned so far in the academic year. The statistics of the tests mean scores, and standard deviations for
both groups were calculated and are presented in Table 4.1 below. T-test was performed to assess the mean difference between the two groups. Table 4.1 shows the groups statistics and t-test results. The results in the table show that there was no significant difference between the two groups. Therefore, by the time of the experiment, there was no significant difference between the two groups.
Table 4.1: Groups Statistics and T-test Results
GROUP N Mean Std. Deviation t Sig TEST1 Experimental Group 29 13.07 4.978 1.591 .118 significantNot Control Group 25 11.04 4.286 TEST2 Experimental Group 29 12.76 4.673 -.074 .941 significantNot Control Group 26 12.85 3.977 TEST3 Experimental Group 29 12.83 4.929 -.752 .455 significantNot Control Group 26 13.73 3.832 TEST4 Experimental Group 29 12.48 6.045 -1.871 .067 Not significant Control Group 26 15.04 3.649 Setting
The main experiment was conducted at the computer laboratory in Khalifa Secondary school. The principal of the school approved conducting the study in the school. The good relationship between the researcher and the school principal was the main reason to select the school. This had eased the process of conducting the experiment.
Instruments
Four instruments were used in the experiment. • A questionnaire.
• The CAT tool containing the biochemistry lessons.
• Achievement test to measure the students' achievement of the assigned lessons. The researcher prepared the test in cooperation with the chemistry teacher who participated in implementing the experiment in Khalifa School.
• The standardized version of the Sternberg-Wagner Self-Assessment inventory on the Legislative, Executive, and Judicial Thinking Styles.
The instruments are described below.
Questionnaire
A student questionnaire was developed for the purpose of evaluating the effectiveness of CAT. The questionnaire was divided into two parts, as shown in appendix III and V). The first part consists of 25 items (the questionnaire in the first phase consisted of only 22 items). Four point Likert type scale was used in the questionnaire to evaluate the effectiveness of CAT: 4 for highly agree, 3 for agree, 2 for neutral, 1 for don't agree have been given in order to analyze the data. Although 5 point Likert would have been a lot better and in line with good practice, the outcome would not change if the extra option had been used. There are very few ticks in the "don't agree" choice so extending it to "strongly don't agree" would not have had any effect.
The second part consists of two open questions to gather students’ comments and suggestions about CAT. After the first week of the experiment six students from the control and experimental group were asked to answer the questionnaire to ensure that all items were clear and understandable. Modification was done accordingly. At the end of the experiment students of both groups answered the questionnaire. No clarification was needed. Cronbach's alpha (Elliott & Woodward 2007) was calculated to confirm sufficient reliability of each of the subscales.
In order to explore the hypotheses 2 through 6 the questionnaire was designed to survey the learning experience dimensions as depicted in table 4.2.
Table 4.2: Questions to Explore the Hypotheses
Learning Experience Question Number
Comprehension 2, 3, 6, 9, 19
Passion for Subject 8
Preferred Learning Style 5
Teaching Method 11, 12, 13, 15, 16, 23, 24 Activities and Tasks 1, 4, 10, 14, 18, 20, 22, 25
CAT Tool
The tool was given the name CAT (Computer Aided Thinking) because it was intended to aid students in their learning by matching activities to their thinking styles. CAT was developed using the programming language Visual Basic 6. It was chosen for the following reasons:
• It is easy to use on one hand and sufficiently powerful on the other to satisfy all the development needs in this study.
• User interfaces can be easily programmed.
• The researcher has considerable previous experience in this language.
Details of CAT are described in chapter 7.
Achievement Test
The researcher prepared an achievement test with the cooperation of the subject teacher. The test was designed to measure the achievement of the individuals who constituted the sample of the study after they had finished studying the content of the assigned material. The test duration was 45 minutes. The test included multiple choices, open ended, and true and false questions. The paper-based test is given in Appendix VI. In order to confirm the validity of the test, it was presented to six specialist evaluators who were specialized in methods of teaching chemistry and in measurement and assessment. The evaluators were asked to offer comments on the wording of the test and the coverage of the test. The final test was finalized after taking all the comments into consideration.
Standardized Thinking Style Inventory
The Sternberg-Wagner Self-Assessment Inventory of thinking styles (Sternberg 1997) was standardized to be used in the UAE for identifying the thinking styles among the sample of the study. The inventory is given in Appendix IV. Chapter 5 details the Standardization study.