Chapter 7 LANA-II Evaluation Methodology and Results
7.6 Experiment 2 – LANA-II engine effectiveness
7.6.4 Comparative descriptive analysis of data from autistic participants and
This section presents further descriptive analysis of the collected data through the log file. This data was collected from all autistic students (30 participants) and all neurotypical students (24 participants). Both participants used LANA-II in both tutorials (Solar system, Electrical circuit and the magnetism). The analysis of results presented in this part are intended to highlight differences of the results between autistic participants and neurotypical participants. The Man-Whitney U test was used in order to highlight any statistically significant differences in the data. The findings are as follows:
7.6.4.1 Duration of conversation whilst engaged with tutorial
Table 7.34 illustrates the results of a Mann-Whitney U test conducted to test whether there was a difference in the time taken to complete the tutorial between the two groups participants.
Table 7.34: Mann-Whitney U test for duration of conversation between the two groups participants
Ranks
Participants N Mean Rank Sum of Ranks Average conversation duration (mins) NT 24 12.50 300.00 Autistic 30 39.50 1185.00 Total 54 Test Statisticsa
Average conversation duration (mins)
Mann-Whitney U .000
Wilcoxon W 300.000
Z -6.866
Asymp. Sig. (2-tailed) .000
a. Grouping Variable: Participants
The results show that there is a statistically significant difference in the time taken to complete the tutorial between the two groups participants (reported by the SPSS as p
133
= .000). When the mean ranks are compared, the results show that autistic participants rank higher than the neurotypical participants, indicating that the autistic participants spent time more than the neurotypical participants to complete the tutorial. The effect size (r) is calculated using the (Equation 2) and found that r = 0.9 indicating a very large effect size using (Cohen, 1988) criteria.
7.6.4.2 Number of unrecognised utterances
Table 7.35 shows the results of the Mann-Whitney U test to determine if there was any difference in number of unrecognised utterances between the two groups participants.
Table 7.35: Mann-Whitney U test for the number of unrecognised utterances between the two groups participants
Ranks
Participants N Mean Rank Sum of Ranks Number of unrecognised utterance NT 24 31.63 759.00 Autistic 30 24.20 726.00 Total 54 Test Statisticsa
Number of unrecognized utterance
Mann-Whitney U 261.000
Wilcoxon W 726.000
Z -2.002
Asymp. Sig. (2-tailed) .045
a. Grouping Variable: Participants
The results show that there is a statistically significant difference in the number of unrecognized utterances between the two groups participants (p = .045). When the mean ranks are compared the results show that neurotypical participants ranks higher than the autistic participants, indicating that the number of unrecognised utterances from neurotypical participants was more than the autistic participants. The effect size (r) is calculated using the (Equation 2) and found that r = 0.3 indicating a medium effect size using (Cohen, 1988) criteria.
7.6.4.3 Number of correct responses
Table 7.36 illustrate the results of the Mann-Whitney U test conducted in order to measure whether there was a statistically significant difference between the numbers of correct responses between the two groups participants.
134
Table 7.36: Mann-Whitney U test for the number of correct utterances between the two groups participants
Ranks
Participants N Mean Rank Sum of Ranks Number of correct response NT 24 22.81 547.50
Autistic 30 31.25 937.50 Total 54
Test Statisticsa
Number of correct response
Mann-Whitney U 247.500
Wilcoxon W 547.500
Z -2.560
Asymp. Sig. (2-tailed) .010
a. Grouping Variable: Participants
From this result, it can be concluded that the number of correct responses between the two groups participants was statistically significant (p = .010). In comparing the mean ranks the autistic participants rank higher, which indicates that the autistic participants having more correct responses than the neurotypical participants. The effect size (r) is calculated using the (Equation 2) and found that r = 0.3 indicating a medium effect size using (Cohen, 1988) criteria.
7.6.4.4 Number of incorrect responses
Table 7.37 outlines the results of a Mann-Whitney U test conducted to test if there was a statistically significant difference between the numbers of incorrect responses between the two groups participants.
Table 7.37: Mann-Whitney U test for the number of incorrected utterances between the two groups participants
Ranks
Participants N Mean Rank Sum of Ranks Number of incorrect response NT 24 32.38 777.00 Autistic 30 23.60 708.00 Total 54 Test Statisticsa
Number of incorrect response
Mann-Whitney U 243.000
Wilcoxon W 708.000
Z -2.625
Asymp. Sig. (2-tailed) .009
a. Grouping Variable: Participants
The results indicated that there were statistically significant differences between the numbers of the incorrect response between the autistic participants and the neurotypical participants (p = 0.009). When comparing the mean ranks it can be seen that the neurotypical participants ranks higher than autistic participants, meaning the
135
neurotypical participants having more incorrect responses than the autistic participants. The effect size (r) is calculated using the (Equation 2) and found that r = 0.4 indicating a nearly large effect size using (Cohen, 1988) criteria.
7.6.4.5 Number of utterances using Similarity Strength
Table 7.38 shows the results of a Mann-Whitney U test to test if there was a statistically significant difference between the numbers of utterances using similarity strength between the two groups participants.
Table 7.38: Mann-Whitney U test for the numbers of utterances using similarity strength. Ranks
Participants N Mean Rank Sum of Ranks Number of utterances using
Similarity Strength
NT 24 39.54 949.00
Autistic 30 17.87 536.00
Total 54
Test Statisticsa
Number of utterances using Similarity Strength
Mann-Whitney U 71.000
Wilcoxon W 536.000
Z -5.327
Asymp. Sig. (2-tailed) .000
a. Grouping Variable: Participants
From this result, it can be summarized that there is a statistically significant difference between the two groups participants (reported by the SPSS as p = .000). When comparing the mean ranks it can be seen that the neurotypical participants ranks higher than autistic participants, meaning that the number of utterances used the similarity strength in neurotypical participants was more than the number in autistic participants, which means most of the autistic student utterances were matched the scripted patterns. The effect size (r) is calculated using the (Equation 2) and found that r = 0.7 indicating a very large effect size using (Cohen, 1988) criteria.