Reliability

In this study split half reliability was used repeatedly to verify the degree to which the instruments yield consistently the same results after the repeated trials according to Mugenda, O. and Mugenda (2003). The researcher had developed four instruments as follows: TAC Effectiveness Questionnaire for Teachers; TAC Effectiveness Questionnaire for TAC tutors; TAC Effectiveness Questionnaire for DEOs and TAC Effectiveness classroom observation schedule. In each case, before use of the instruments for pilot testing, split half reliability was established through manual calculations because data were manually manageable and then after pilot study reliability was verified through assistance of Statistical Package of Social Sciences (SPSS) in the revised instruments for the actual study. Split half reliability was found appropriate because it is a measure of internal consistency of a test according to Gay, Mills and Airisia (2009) who argued that the odd even strategy for splitting test works out well regardless of test organization. In addition Mugenda, O and Mugenda (2003) explained that the major advantage of split half approach is that it eliminates the chance error due to differing test conditions as in test retest and equivalent form techniques and that data with split half reliability have a high correlation coefficient. This study therefore chose split half approach.

97

The items in each instrument used in this study were randomly split into two equal sets using even and odd numbers strategy to come up with two equal sets of items. This was in an attempt to randomize and standardize them. All items bearing even numbers made a set and items bearing odd numbers made another set. However, splitting the items was not applied to classroom observation schedule where two observers scored and came up with two sets of scores. Split half reliability for each questionnaire was established following the six steps procedure provided by Gay, Mills and Airisia (2009). The procedure for establishing split half reliability followed was as follows: total items in the instrument were administered; the items were then divided into two halves by selecting the odd items to make one sub test and the even items made another sub test - questionnaire; each participant’s score on each sub test- questionnaire was computed. The two sets of scores were correlated using Spearman rho coefficient of correlation or rank order

following the formula: rho=1-6∑d to establish reliability of half test. n(n2-1)

Reliability coefficient was then established for total test using Spearman Brown Prophesy formula: rtotal test=2(r split half)

1+(r split half).

To establish reliability of the TAC Effectiveness Questionnaire for teachers, the whole questionnaire was administered to the respondents. Each item was scored out of a total score of three. Clarity and relationship to objectives and research question scored two and reasonable response was scored one in each

98

item. Then 20 items which are divisible by two out of the 21 open ended items were used to establish the reliability. The items were randomly split into two equal sets along even and odd numbers to come up with two equal tests of 10 items each. The two sets of scores were correlated using Spearman rank order correlation coefficient (rho) for split half and then Spearman Brown Prophecy formula coefficient for total test. The questionnaire scored .86 correlation coefficient. The instrument was then used for pilot study. According to Plowright (2011) a reliability of at least .5 correlation coefficient is acceptable hence the instrument was acceptable. While Kerlinger (2000) said that .7 correlation coefficient and above was acceptable, hence the instrument was accepted. The final data from the actual study was collected and coded and ran on SPSS for reliability correlation coefficient using Spearman – Brown and the correlation coefficient was .75 correlation coefficient

In TAC Effectiveness Questionnaire for TAC tutors 28 items were used. They were split into two and each set had 14 items. The same procedure proposed by Gay, Airisia and Mills was followed using rho and Spearman-Brown Prophecy formula and it yielded .87correlation coefficient. The instrument was used in the pilot study. The items used in the actual study yielded .77 correlation coefficient when correlated with the aid of SPSS.

The TAC Effectiveness Questionnaire for DEO had six items which were split into two and the same process was used as in the other questionnaires. The

99

questionnaire yielded.86 correlation coefficient and was used in the pilot study. When data were collected and reliability was calculated with aid of SPSS using Spearman-Brown Coefficient the data yielded .75 correlation of coefficient.

To establish correlation coefficient of the TAC Effectiveness classroom observation schedule, two observers discussed the items and agreed to follow the given criteria for scoring the items. Kerlinger (2000) argued that reliability of an observation system is determined by agreement among observers. As in the other instruments, the observers awarded each item out of a maximum of three scores. Their observation was guided by an observation schedule. Two scores were based on the clarity of the item and its adherence to the objectives and the research questions and one mark was awarded to each item by each researcher based on their own fair judgment of the situation observed. They came up with two sets of scores for each of the seven items. The results by the two observers were correlated as advocated by Kerlinger (2000) using Spearman rho and then Spearman Brown Prophesy formula and yielded .85 correlation of coefficient. The instrument was then used in the actual study. Actual research data were not again tested for reliability because only the researcher conducted classroom observation in the actual research data. People were not willing to co - observe because the exercise was tedious, costly and time consuming.

100

The researcher settled on the SPSS assisted calculations of correlation coefficient except for the TAC Effectiveness Observation schedule which retained the initially established reliability. The reliability of the four instruments was therefore as follows: TAC effectiveness questionnaire for teachers .755, TAC effectiveness questionnaire for TAC tutors .772 correlation coefficient, TAC effectiveness questionnaire for DEOs .751correlation coefficient and TAC effectiveness classroom observation schedule.85 correlation coefficient

In conclusion, use of questionnaire and observation schedule, proof reading by some QASO and TAC tutors, KIE officers, teachers, colleagues from Maasai Mara University and the pilot testing; all as recommended by Gall and Borg (1983), enhanced the reliability of the research instruments.

3.6.7 Piloting

The instruments were pilot tested on a small representative sample of the target population outside the five counties used in the main study. The methodology used in the actual study was followed also in the pilot study which employed purposive sampling and simple random sampling to select the study sample and the respondents. Purposive sampling was appropriate to select targeted specific subjects for the study, who met specified criteria as was advocated by Gay, Arisia & Mills (2009). In selecting the counties, the specific criteria was Kenya Certificate of Primary Education (KCPE) results representing poor

101

performance, mediocre, average and above average, covering the period 2007- 2011 used to select counties. These had to be selected purposively. The counties selected were the following in the grades indicated: Isiolo - poor; Narok - mediocre; Kajiado- average and Elgeiyo Marakwet - above average. The pilot sample selected was 20 respondents who filled in the TAC Effectiveness questionnaire for teachers, four TAC tutors filled in TAC Effectiveness Questionnaire for TAC tutors and two DEOs filled in the TAC Effectiveness Questionnaire for DEOs.

The instruments were administered to the respondents who were encouraged to explain whether the items contained complicated, difficult or ambiguous words, unclear questions, errors, sensitive issues, and also to comment on length of the instruments, the space provided for writing in the questionnaire and the time provided to fill in the questionnaires. The respondents gave their suggestions on areas of the instruments to be revised and their comments were used to improve the instruments. The pilot report therefore was used to make corrections on the instruments before use in the study and to ensure that the exercise yielded valid and reliable data. Errors on use of words, concepts and statements which could be misconstrued and cause misinterpretation and bias were identified and corrected. For example, in the questionnaire for TAC tutors question one, the word ‘means’ was substituted for ‘criteria’; ‘length of time as TAC tutor’ was changed to ‘work experience’; ‘topic discussed’ was changed to ‘areas facilitated’; areas to increase writing space were indicated; rating as

102

poor; fair, very good was changed to very high, high, average, low, very low. Suggestion was made that the duties of TAC tutors should be listed for respondents to tick rather than writing them down with no reference. Repeated items were deleted.

The pilot data were analyzed using descriptive statistics with help of the Statistical Package of Social Sciences (SPSS) and the results critiqued to determine whether they reflected the intended study objectives. The results revealed that the effectiveness of TACs in teacher professional development had gaps as indicated in the research problem. For example regarding areas and topics covered in in-service, 11(55%) teachers out of 20 reported they were not relevant. A total of 13 (65%) teachers out of 20 reported that issues discussed after classroom observation were not relevant to TPD. TAC resources were viewed as the greatest challenge with 12(60%) teachers suggesting that provision of resources would be a measure to revitalize TACs. The two DEOs used in pilot study confirmed that the TAC tutors were over worked and they had no time to conduct their duties and responsibilities. Teachers observed were reported to have been using teacher centred teaching approaches. The survey research design, instruments, descriptive statistics as a means of data analysis and use of SPSS were found effective and were adapted in the study. The reliability coefficient using responses from teacher questionnaire was calculated using Spearman Brown correlation coefficient ran on SPSS and the

103

result was .75 correlation coefficient and in actual study the correlation coefficient was .75.

Pilot testing aided in establishing content validity of the instruments and improvement of the question format and scales and to make corrections on the instruments (Creswell 2009 & Orodho, 2009).