After finding that CSTutor is effective when learning to program, I wanted to explore more deeply to find any student groups that get the highest benefit from CSTutor. The students who use CSTutor can be separated into two groups: the group of students who have learnt programming before and the group of student who have not. There are 17 students (47%) in the first group, and 19 students (53%) in the second group (see Figure 6.4).
An independent samples one-tailed t-test was performed to see if there is a relationship between the students’ programming background and the results of their:
1. pre-test 2. post-test, and
3. their increase in score.
Figure 6.4 The number of participants grouped by the programming background
The null hypothesis for the first test is:
The average pre-test score of the students with a programming background is not better than the average pre-test score of the students without a programming background.
The result of the independent samples two-tailed t-test of the students’ programming background to the pre-test score is shown in Table 6.10. The p-value of 0.004 in that table can be divided by 2 to obtain the p-value of one-tailed t-test (i.e.: 0.002). The p-value of 0.002 – less than 0.05 – shows strong evidence against the null hypothesis. By looking at the average pre-test scores for both groups of students
53% 47%
Have Learnt Programming Before
No Yes
in Table 6.10 (77.35% for student with a programming background and 52.37% for students without a programming background), I can conclude that the average pre- test score of the students’ with programming background is significantly better than the average pre-test score of the students without a programming background. This result is not surprising. Students with a programming background should be able to answer some programming questions in the pre-test that are similar across multiple programming languages. Therefore their pre-test result should be higher than the students without a programming background.
Table 6.10 The students’ programming backgrounds and their pre-test scores
Group Statistics
LeantProgrammingBefore N Mean Std. Deviation Std. Error of Mean
PreTest
Yes 17 77.35% 14.374% 3.486%
No 19 52.37% 30.703% 7.044%
Independent Samples Test Levene's Test for
Equality of Variances
t-test for Equality of Means
F Sig. t df Sig. (2-tailed) Mean Difference Std. Error Difference 95% Confidence Interval of the Difference Lower Upper Pre- Test Equal variances assumed 7.357 .010 3.065 34 .004 24.985% 8.152% 8.417% 41.552% Equal variances not assumed 3.179 26.14 .004 24.985% 7.859% 8.833% 41.136%
The next independent samples one-tailed t-test evaluates the effect of the students’ programming background on the result of the post-test. The null hypothesis for this test is:
The average post-test score of the students with a programming background is not better than the average post-test score of the students without a programming background.
The two-tailed test in Table 6.11 shows the p-value of 0.150. Therefore, the p- value of one-tailed test is 0.075 (0.150 / 2). Because the p-value is not less than 0.05 then there is no strong evidence to reject the null hypothesis. Therefore the average post-test score of students with a programming background (i.e.:84.12%) is not significantly better than the average post-test score for the students without a programming background (i.e.:76.32%). In other words, the students without a programming background can achieve post-test scores as good as the students with a programming background.
Table 6.11 The students’ programming background and the post-test score
Group Statistics
LeantProgrammingBefore N Mean Std. Deviation Std. Error Mean
PostTest
Yes 17 84.12% 11.890% 2.884%
No 19 76.32% 18.697% 4.289%
Independent Samples Test Levene's Test
for Equality of Variances
t-test for Equality of Means
F Sig. t df Sig. (2-tailed) Mean Difference Std. Error Difference 95% Confidence Interval of the Difference Lower Upper PostT est Equal variances assumed 1.058 .311 1.473 34 .150 7.802% 5.295% -2.960% 18.563% Equal variances not assumed 1.510 30.86 .141 7.802% 5.168% -2.741% 18.345%
The first and the second independent samples one-tailed t-tests above can be used to find out which group gets the highest benefit from CSTutor. However to make the conclusion clear, I performed a third independent samples t-test. This t-test evaluates the effect of the students’ programming background on the students’ increase in score in the post-test. The null hypothesis for this test is:
The increase in score of the students without a programming background is not higher than the increase in score of the students with a programming background.
The p-value of 0.004 in Table 6.12 shows the result of the two-tailed t-test. The p-value of one-tailed t-test is 0.002. This value (that is less than 0.05) shows strong evidence to reject the null hypothesis. The average increase in score for both groups of students (23.95% for students without a programming background and 6.76% for students with a programming background) shows that the increase in score of students without a programming background is significantly higher than the increase in score of students with a programming background.
The results from Table 6.12 show that the students who get the highest benefit from CSTutor are the students who have not learnt programming before. The chart in Figure 6.5 gives a visual representation of this effect.
Table 6.12 The students’ programming background and the increase in score
Group Statistics
LeantProgrammingBefore N Mean Std. Deviation Std. Error Mean
Increase inScore
Yes 17 6.76% 10.449% 2.534%
No 19 23.95% 21.054% 4.830%
Independent Samples Test Levene's Test
for Equality of Variances
t-test for Equality of Means
F Sig. t df Sig. (2-tailed) Mean Difference Std. Error Difference 95% Confidence Interval of the Difference Lower Upper Increase inScore Equal variances assumed 4.375 .044 -3.043 34 .004 -17.183% 5.646% -28.658% -5.708% Equal variances not assumed -3.150 26.98 .004 -17.183% 5.455% -28.375% -5.990%
Figure 6.5 The effect of the student programming background to the student score 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Pre-Test Average Post-Test Average Increasing Score Average S tu d e n t S co re
The Effect of the Student Programming
Background on the Student Score
Have programming background
Do not have programming background