Post-Experiment Questionnaire

The post-experiment questionnaire consisted of ten questions to which the participants responded using a 7 point Likert scale. An answer of 1 indicated an answer of “very much so” to the statement and an answer of 7 indicated an answer of “not at all”. The first seven questions are discussed next, followed by an analysis of the last three questions, which asked the users if they considered each of the three interface conditions as the best interface.

ˆ Q1Do you feel the system reacted the way you thought it would before you

began the experiment? The mean response for this question was 3.40 (SE

0.62). A two tailed t-test showed that there was no significant difference from the mid point value of 4, with p = 0.36. Therefore, although the mean of the participants answers indicated that the participants felt the system reacted as expected, the result was not significantly different from a neutral response.

ˆ Q2 How well did you feel the system understood your verbal spatial ref-

erences? The mean response for this question was 1.60 (SE 0.16). A

two tailed t-test showed that there was significant difference from the mid point value of 4, with p< 0.05, indicating that the participants felt strongly that the system did understand the spatial language that they used.

ˆ Q3 How well did you feel the system understood the gestures you used?

94

t-test showed that there was no significant difference from the mid point value of 4, with p = 0.26. Therefore, although the mean of the partici- pants answers indicated that the participants felt the system understood their gesturing, the result was not significantly different from a neutral response.

ˆ Q4 How well did you feel the system reacted the way you wanted it to?

The mean response for this question was 2.90 (SE 0.48). A two tailed t- test showed that there was significant difference from the mid point value of 4, with p < 0.05, indicating that the participants felt the reaction of the system was what they had intended it to be.

ˆ Q5 Do you feel the use of gestures helped you communicate spatially with

the system? The mean response for this question was 3.40 (SE 0.50) .

A two tailed t-test showed that there was no significant difference from the mid point value of 4, with p = 0.26. Therefore, the result was not significantly different from a neutral response.

ˆ Q6 Did you have confidence speaking to the system? The mean response

for this question was 2.20 (SE 0.59). A two tailed t-test showed that there was significant difference from the mid point value of 4, with p

< 0.05, indicating that the participants felt strongly that the system responded well to verbal input.

ˆ Q7Did you have confidence gesturing to the system? The mean response

for this question was 3.50 (SE 0.59). A two tailed t-test showed that there was no significant difference from the mid point value of 4, with p = 0.43. Therefore, although the mean of the participants answers indicated that the participants felt that the system responded well to gesture input, the result was not significantly different from a neutral response.

The results of the post experiment questionnaire can be seen in Table 6.3. These results show that participants felt the system understood verbal spatial references very well and that the system reacted the way they expected it to. Users also felt comfortable speaking to the system. Responses for how well the users felt the system understood gestures were neutral. This result should be expected since the wizard was able to fully understand the speech used, but had to interpret the gestures, which took time and, when gestures were ambiguous, the wizard was not always able to understand them.

Mean Std Error

T-Test (p)

Q1: Do you feel the system reacted the way you thought it would before you began the experiment?

3.40 0.62 0.36

Q2: How well did you feel the system understood your verbal spatial references?

1.60 0.16 <0.05

Q3: How well did you feel the system understood the gestures you used?

3.30 0.58 0.26

Q4: How well did you feel the system reacted the way you wanted it to?

2.90 0.48 <0.05

Q5: Do you feel the use of gestures helped you com- municate spatially with the system?

3.40 0.50 0.26

Q6: Did you have confidence speaking to the system? 2.20 0.59 <0.05 Q7: Did you have confidence gesturing to the system? 3.50 0.59 0.43

Table 6.3 Summary of questionnaire responses. Questions were posed on a seven point Likert scale between 1 = Very Much So and 7 = Not at All.

The final three questions asked the participants which of the interfaces they felt was the best, with responses given on a Likert scale from 1 (very much so) to 7 (not at all). The results are shown in Figure 6.13. An ANOVA test resulted in (F2,27 = 4.09, p <0.05) showing there was a significant effect

due to condition. Pairwise comparison using Bonferroni correction (p <0.05) revealed significant differences between the multimodal (Speech and Gesture) and the Gesture Only conditions. However, there was no significant difference between the Speech and other two conditions. Therefore, although the mean of the participants answers indicated that the participants felt the Speech and Gesture condition was the best, the result was not significantly significant.