The purpose of this chapter is to analyse the quantitative questionnaire data. This section contains the statistical analysis, as discussed in the sections above. As mentioned above, the independent variables include participants’ age, occupation, institution, gender, familiarity with teaching and leaning online, and computer literacy. The dependent variables are divided into four themes, each aligning to four of the research objectives. They include the Analysis of Perspectives of Teaching and Learning, Analysis of User-friendly Design, Analysis of Learner-friendly Design, and Analysis of Behaviourist and Constructivist-Oriented Design.
The dependent variables in the Analysis of Perspectives of Teaching are divided into one sub-theme: Online versus Face-to-Face instruction. The dependent variables in the Analysis of User-friendly Design are divided into two sub-themes: Navigation; and, Usability and Screen Design. The dependent variables in the Analysis of Learner-friendly Design are divided into three sub-themes: Learner Consultation; Learner Styles; and, Learner Design Features. The Analysis of Behaviourist and Constructivist Design is divided into eight categories: Learner Participation; Direct Instruction; Open-ended; Recall and Memorisation; Learner Objectives; Real World Learning and Higher Order Thinking; Multiple-choice; and, Collaboration. These sub-themes and their relating questionnaire items (dependent variables) are also displayed earlier in this chapter, in Table 5.2.1.
5.5.1 Analysis of Perspectives of Teaching and Learning
The questionnaire was based on Likert’s Measurement of Attitudes (1932), from 1 (STRONGLY AGREE), 2 (AGREE), 3 (NOT SURE), 4 (DISAGREE), and 5 (STRONGY DISAGREE). There was one sub-theme – Online versus Face-to-Face interaction, and three different questions relating to teaching and learning online (Q.18, Q.27, and Q.29). The following analysis discusses the Mean score according to participants’ occupation. It then investigates whether a statistically significant variation of opinion occurs between each of the independent variables with more than two groups (occupation, age, institution, familiarity with teaching and learning online, computer literacy). If a statistically significant variation of opinion is found, Tukey’s post hoc test will be performed to establish where the significant relationships have occurred. Finally, an independent T-Test analysis will be performed, for independent variables with two groups (gender). A discussion of the findings follows within the next section.
5.5.1.1 Online Versus Face-to-Face Learning
Table 5.5.1.1a: Q.18 Means of question ‘Online chat is an effective alternative to face-to-face learning’
Q.18 Online chat is an effective alternative to face-to- face learning Occupation Means academic_fe 3.13 academic_od 3.50 unistudent_fe 2.89 unistudent_od 2.88 professional_id 3.70 professional_wd 3.50 Total 2.98
The overall Mean score for participants is 2.98, which shows that generally participants are closer to being not sure, in regards to whether online chat is an effective alternative to face- to-face learning. There is a noticeable variation of opinions within the groups by occupation. Students from Other Disciplines (2.88) and Education (2.89) have the lowest Mean scores, and generally agree that online chat is an effective alternative to face-to-face participation. Academics and professionals generally are not sure whether online chat is an effective alternative to face-to-face participation. Professional instructional designers have the highest Mean score (3.70), and there is noticeable difference of opinion between this group and that of students. The analysis of ANOVA for whether online chat is an effective alternative for face to-face learning shows that there is no statistically significant difference between groups by age, occupation, institution, familiarity with teaching and learning online or computer literacy. The Independent T-Test Sample analysis shows that there is no statistically significant difference in the means according to gender.
Table 5.5.1.1b: Means of Q.27 ‘Online participation increases my motivation more than face- to-face’
Q.27 Online participation increases my motivation more than face-to-face participation Occupation Means academic_educ 3.48 academc_otherdisc 3.73 student_educ 3.04 student_otherdisc 3.22 professiona_id 3.80 professional_wd 3.75 Total 3.22
The overall Mean score for participants is 3.22, which shows that generally participants are not sure, in regards to whether online participation increases their motivation more face-to- face learning. Whilst all the above groups have a Mean score located between not sure and disagree, there is still a noticeable difference of opinion amongst participants. Students from Education (3.04) have the lowest Mean score, followed by students from Other Disciplines (3.22), and academics from Education (3.48). The other occupational groups have higher Mean scores, professional instructional designers (3.80) have the highest Mean score, and there is a noticeable difference of opinion between this group and that of students from Education. The analysis of ANOVA for whether online participation increases motivation more than face-to-face participation shows that there is significant difference between participants according to age. The table representing significant difference according to age is shown in Table 5.5.1.1c below:
Table 5.5.1.1c: ANOVA for Q.27 by Age
Sum of
Squares df Mean Square
statistical significance
Between Groups 17.224 3 5.741 .003
Within Groups 286.677 237 1.210
Total 303.900 240
In order to discover which groups show a statistically significant difference, a post hoc test using Tukey’s HSD is used. The following result indicates a statistically significant difference between two age groups:
Participants aged 18-29 and 40-49 (0.03)
Participants from the youngest age group 18-29 (3.04) are significantly keener than those aged 40-49 (3.95), in regards to whether online participation increases their motivation more than face-to-face participation. The Independent T-Test Sample analysis shows that there is no statistically significant difference in the Means according to gender.
Table 5.5.1.1d: Means of Q.29 ‘Courseware should be used in addition to face-to-face interaction’
Q.29 Courseware should be used in addition to face-to-face interaction
Occupation Means academic_educ 1.87 academc_otherdisc 2.42 student_educ 2.10 student_otherdisc 1.98 professiona_id 2.40 professional_wd 2.50 Total 2.07
The overall Mean score for participants is 2.07, which shows that generally participants agree that courseware should be used in addition to face-to-face interaction. There is a noticeable variation of opinions within the groups by occupation. Academics from Education (1.87) have the lowest Mean score of all groups, and strongly agree that courseware should be used in addition to face-to-face interaction. Students’ low Mean scores indicate they are somewhat keener than academics from Other Disciplines and professionals in regards to whether courseware should be used in addition to face-to-face interaction. Professional web developers have the highest Mean score, and there is a noticeable difference of opinion between this group and academics from Education. The analysis of ANOVA for whether courseware should be used in addition to face-to-face interaction shows that there are no statistically significant differences between groups by age, occupation, institution, familiarity with teaching and learning online or computer literacy. The Independent T-Test Sample
analysis shows that there is no statistically significant difference in the Means according to gender.
5.5.2 Analysis of User-Friendly Design
The questionnaire was based on Likert’s Measurement of Attitudes (1932), from 1 (STRONGLY AGREE), 2 (AGREE), 3 (NOT SURE), 4 (DISAGREE), and 5 (STRONGY DISAGREE). There were two categories– Navigation, and Usability and Screen Design. There were three questions relating to Navigation one slightly different (Q.5) and two that were similar (Q.7 and Q.20). These questions were grouped together under the name ‘nav_inst’. There were three different questions relating to Usability and Screen Design (Q.21, Q.24, and Q.25). The following analysis discusses the Mean score according to participants’ occupation. It then investigates whether a statistically significant variation of opinion occurs between each of the independent variables with more than two groups (occupation, age, institution, familiarity with teaching and learning online, computer literacy). If a statistically significant variation of opinion is found, Tukey’s post hoc test will be performed to establish where the significant relationships have occurred. Finally, an independent T-Test analysis will be performed, for independent variables with two groups (gender). A discussion of the findings follows within the next section.
5.5.2.1 Navigation
Table 5.5.2.1a: Means of Q.5 ‘An open-ended learning environment should be present in courseware’
Q.5 An open-ended learning environment should be present in a courseware Occupation Means academic_educ 1.87 academc_otherdisc 2.36 student_educ 2.11 student_otherdisc 2.30 professiona_id 3.00 professional_wd 2.75 Total 2.22
The overall Mean score for participants is 2.22, which shows that generally participants agree that an open-ended learning environment should be present in a courseware. There is a significant variation of opinions within the groups by occupation. Academics from Education (1.87) have the lowest Mean score, and strongly agree that an open-ended learning environment should be present in courseware. The Mean score of most of the other occupational groups is located between agree and not sure, and students from Education (2.11) were slightly keener than other groups here. Professional Instructional Designers had
the highest Mean score, and were the only group who weren’t sure whether an open-ended learning environment should be present in courseware.
The analysis of ANOVA on whether an open-ended learning environment should be present in courseware did show there is significant different amongst groups according to occupation and institution. The ANOVA Tables 5.5.2.1b and 5.5.2.1c shows the variance according to occupation, and then institution.
5.5.2.1b: ANOVA for Q.5 by occupation
Sum of
Squares df Mean Square
statistical significance
Between Groups 12.153 5 2.431 .017
Within Groups 203.288 237 .858
Total 215.440 242
5.5.2.1c: ANOVA for Q.5 by institution Squares Sum of df Mean Square significance statistical
Between Groups 7.790 2 3.895 .012
Within Groups 207.650 240 .865
Total 215.440 242
ANOVA does not specify the groups, so a post hoc test is necessary in order to identify the group. TUKEY HSD was chosen for this assignment. The results show that for Q.5, the following pairs show a significant relationship:
- Academics from Education/professional instructional designers (0.018)
- Professional instructional designers/students from Education (0.046)
- Participants from UTAS/professional training providers (0.021)
Academics and students from Education are significantly keener than professional instructional designers; and participants from UTAS (2.15) are also significantly keener than those from professional training providers (3.00), in regards to whether an open-ended learning environment should be present in courseware. As a majority of UTAS participants are from Education, and the majority of professionals were instructional designers, then this significant relationship is perhaps a reflection of the statistical analysis by occupation, for this question. An Independent Samples T-Test is required to assess whether there is significant
difference, for independent variables with two or less sub-groups in this case gender. The
analysis found there is a statistically significant difference between males and females, of 0.000 where significance is calculated at 0.05. Females (2.18) are significantly more confident with an open-ended learning environment than males (2.27).
Table 5.5.2.1d: Means for Q.7, Q.20 and nav_inst (Q.7 and Q.20) Q. 7 Direct instructional guidance to using the courseware is essential Q.20 Learners should be given clear navigational directions when using the courseware nav_inst
Occupation Means Means Means
academic_educ 2.13 1.52 1.8261 academc_otherdisc 2.17 1.50 1.8333 student_educ 2.30 1.97 2.1386 student_otherdisc 2.18 1.94 2.0652 professiona_id 2.80 1.50 2.1500 professional_wd 2.50 1.75 2.1250 Total 2.26 1.87 2.0661
The overall Mean score for nav_inst (Q.7 and Q.20) is 2.061, which shows that generally participants agree that learners’ should be given clear and direct guidance to using the courseware’s navigation. There is little difference in the overall Mean scores by occupation for nav_inst, which indicates that overall participants are in agreement. The overall Mean score for Q.7 (2.26) is slightly higher than that for Q20 (1.97). Generally, participants believe that providing learners with ‘clear navigational directions’ is more important than providing ‘direct instructional guidance’ within courseware. This occurrence may occur because Q.7 is worded as ‘direct instructional guidance’ and might have been interpreted as either instructional or navigational assistance, whereas Q.20 refers specifically to ‘clear navigation’. There is notable variation of opinion within the groups by occupation for Q.7, where both academics from Education and Other Disciplines are much keener than professional instructional designers on the use of direct instructional guidance within courseware. There is also a slight variation of opinion within the groups by occupation for Q.20, where clear navigational directions in courseware are more important to academics and professional instructional designers, than students. An analysis of ANOVA shows that there is significant difference of opinion between the groups according to age, for both for Q.20 and overall for nav_inst. The ANOVA Table 5.5.2.1e shows the variance:
Table 5.5.2.1e: ANOVA for Q.20
Sum of
Squares df Mean Square
statistical significance Q.20 Learners should be
given clear navigational directions when using the courseware
Between Groups
15.351 3 5.117 .000
Within Groups 184.435 239 .772
Total 199.786 242
nav_inst Between Groups 9.056 3 3.019 .002
Within Groups 138.387 238 .581
Total 147.442 241
ANOVA does not specify the groups, so a post hoc test is necessary in order to identify the group. TUKEY HSD was chosen for this assignment. The results show that the following pairs show a significant relationship for Q.20 according to age:
- Participants aged 18-29/ Participants aged 30-39 (0.028)
- Participants aged 18-29/ Participants aged 40-49 (0.009)
- Participants aged 18-29/ Participants aged 50-59 (0.035).
Clear navigational directions in using courseware are significantly more important to older participants aged 30-39 (1.59), 40-49 (1.41), and 50-59 (1.51), than those aged 18-29 (2.04). The following pairs show a significant relationship for nav_inst:
- Participants’ aged 18-29/Participants aged 30-39 (0.06).
Participants aged 30-39 (1.7432) consider clear and direct guidance to using the courseware’s navigation significantly more important than those from the youngest age group 18-29 (2.2019). An Independent Samples T-Test is required to assess whether there is significant difference, for independent variables with two or less sub-groups – in this case gender. The analysis found there is a statistically significant difference between males and females for Q.7 of 0.015 where significance is calculated at 0.05. The inclusion of clear and direct guidance to using the courseware’s navigation is significantly more important to females (2.17) than to males (2.39).
5.5.2.2 Usability and Screen Design
Table 5.5.2.2a: Q.21 ‘Courseware should utilise effective usability, e.g. clear navigation, and good screen design, help menu’
Q.21 Courseware should utilise effective usability, e.g. clear navigation and good screen design, help menu
Occupation Means academic_educ 1.30 academc_otherdisc 1.17 student_educ 1.76 student_otherdisc 1.66 professiona_id 1.30 professional_wd 2.00 Total 1.63
The overall Mean score for participants is 1.63, which shows that generally participants strongly agree that courseware should utilise effective usability (e.g. clear navigation and good screen design, help menu). There is noticeable variation of opinion within the groups by occupation. The inclusion of effective usability in courseware is most important to academics from Other Disciplines (1.17) whom are the occupational group with the lowest Mean score. Effective usability is equally important to academics from Education and professional instructional designers (1.30), whom also have low Mean scores. Students from Other Disciplines (1.66) and Education (1.76) have slightly higher Mean scores than the previous groups, although they still strongly agree that effective usability is important in courseware. Web developers (2.0) are the only group with a Mean score located at agree as opposed to strongly agree. The inclusion of effective usability in courseware is notably less important to this group, than other groups by occupation. The analysis of ANOVA for usability shows there is significant different amongst groups according to age, institution, familiarity with teaching and learning online, and computer literacy. The ANOVA Tables 5.5.2.2b, 5.5.2.2c, 5.5.2.2d, and 5.5.2.2e shows the variance according to age, institution, familiarity with teaching and learning online, and computer literacy.
Table 5.5.2.2b ANOVA for Q.21 by age
Sum of
Squares df Mean Square
statistical significance
Between Groups 10.289 3 3.430 .003
Within Groups 168.114 239 .703
Table 5.5.2.2c ANOVA for Q.21 by institution Squares Sum of df Mean Square significance statistical
Between Groups 7.193 2 3.596 .007
Within Groups 171.210 240 .713
Total 178.403 242
Table 5.5.2.2d ANOVA for Q.21 by familiarity with teaching and learning online Squares Sum of df Mean Square significance statistical
Between Groups 8.954 3 2.985 .006
Within Groups 169.450 239 .709
Total 178.403 242
Tale 5.5.2.2e ANOVA for Q.21 by computer literacy
Sum of
Squares df Mean Square
statistical significance
Between Groups 6.267 2 3.133 .014
Within Groups 172.136 240 .717
Total 178.403 242
ANOVA does not specify the groups, so a post hoc test is necessary in order to identify the group(s). TUKEY HSD was chosen for this assignment. The results show that the following pairs show a significant relationship:
- Participants aged 18-29/Participants aged 30-39 (0.029)
- Participants aged 18-29/Participants aged 50-59 (0.042)
- Participants from UTAS/Participants from professional training providers (0.038)
- Participants with little familiarity with online teaching and learning/Participants with
much familiarity (0.012)
- Participants with average familiarity with online teaching and learning/Participants
with much familiarity (0.007)
- Participants with high levels of computer literacy/Participants with intermediate levels
of computer literacy (0.026)
Effective usability within courseware is significantly more important to participants aged 30- 39 (1.35) and 50-59 (1.27) compared to those from the youngest age group 18-29 (1.78); participants from professional training providers (1.00) compared to participants from UTAS (1.71); participants with the most familiarity with teaching and learning online (1.29) compared to those with little (1.76) and average (1.74) familiarity with teaching and learning online; and participants with high levels of computer literacy (1.54) compared to those with average (1.88) computer literacy. The Independent T-Test Sample analysis shows that there is no statistically significant difference in the Means according to gender.
Table 5.5.2.2f: Q.24 ‘Attractive screen design enhances my motivation to learn’ Q.24 Attractive screen design enhances
my motivation to learn. Occupation Means academic_educ 2.00 academc_otherdisc 1.58 student_educ 2.09 student_otherdisc 2.30 professiona_id 1.90 professional_wd 2.25 Total 2.13
The overall Mean score for participants is (2.13), which shows that generally participants agree that attractive screen design enhances their motivation to learn. There is noticeable variation of opinion within the groups by occupation. Academics from Other Disciplines have the lowest Mean score (1.58). Along with professional instructional designers (1.90) they strongly agree that attractive screen design enhances learning motivation. Whilst the other occupational groups consider attractive screen design important to learning motivation, their Mean score is higher. Students from Other Disciplines (2.30) have the highest Mean score, and there is noticeable difference of opinion between this group and that of academics from Other Disciplines. From the analysis of ANOVA, there is statistically significant difference amongst participants according to their level of computer literacy. The ANOVA Table 5.5.2.2g shows the variance:
Table 5.5.2.2g: ANOVA of Q.24
Sum of
Squares df Mean Square
statistical significance
Between Groups 7.011 2 3.506 .038
Within Groups 249.984 237 1.055
Total 256.996 239
ANOVA does not specify the groups, so a post hoc test is necessary in order to identify the group(s). TUKEY HSD was chosen for this assignment. The results show that the following pairs show a significant relationship for Q.24:
- Participants with high levels of computer literacy/Participants with intermediate levels
of computer literacy (0.029).
Participants with the highest levels of computer literacy (2.03) believe that attractive screen design is significantly more important in terms of learning motivation compared to those with average levels of computer literacy (2.44). The Independent T-Test Sample analysis shows that there is no statistically significant difference in the Means according to gender.
Table 5.5.2.2h: ‘Screen design and layout affect my ability to use the courseware’ Q.25 Screen design and layout affects
my ability to use the courseware
Occupation Means academic_educ 1.83 academc_otherdisc 1.50 student_educ 2.07 student_otherdisc 1.97 professiona_id 1.70 professional_wd 2.00 Total 1.96
The overall Mean score for participants is 1.96, which indicates that overall participants are located between strongly agree and agree in regards to whether screen design and layout affects their ability to use the courseware. There is slight variation of opinion between the groups by occupation. Academics from Other Disciplines (1.50) have the lowest Mean score, and strongly agree that screen design and layout affects usability. Professional instructional designers (1.70) and academics from Education (1.83) also strongly agree. The Mean score of students from Other Disciplines (1.97) is located between strongly agree and agree. Professional Web Developers (2.00) and students from Education (2.07) have higher Mean scores, which indicates that screen design and layout affects these groups less. The analysis of ANOVA for whether screen design and layout affects their ability to use the