After the introductory demographic questions, the first main section of the survey included two questions related to faculty perceptions of online courses in general. The first question consisted of sixteen five-point Likert items based on a survey instrument developed by Osborne et al. (2009) and modified to pertain to the study topic of engineering education. Many of these questions or variations were also used in the pilot study (Kinney, Liu, and Thornton, 2012). The second question consisted of six items pertaining to respondent perceptions of whether certain topics can be effectively taught online.
Table 8
General Perception Questions – Descriptive Statistics
Question Mean Standard
Deviation
Online courses in engineering are easier for students than face-to-face courses.
2.49 0.981 Online courses in non-engineering topics are
easier for students than face-to-face courses.
3.07 0.884 Learning outcomes are comparable in online and
face-to-face engineering courses.
2.60 1.126 Learning outcomes are comparable in online and
face-to-face non-engineering courses.
2.81 0.998 Students are less willing to 'speak their mind' in an
online class than in a face-to-face class. 3.08 0.945 Students communicate more in an online class
than they do in a face-to-face class.
2.56 0.859 Online courses require more time for students to
complete successfully than face-to-face courses.
2.91 0.883 Face-to-face classes provide better opportunities
for students to interact than online classes.
4.25 0.917 Student and faculty interactions are more effective
in face-to-face classes than they are in online classes.
4.32 0.819 More problems occur in online courses than face-
to-face courses.
3.37 0.849 More students withdraw from online courses than
face-to-face courses.
3.34 0.841 Students who procrastinate should not take an
online course.
3.89 0.929 Students require more discipline to succeed in
online courses.
4.07 0.787 Online courses can be taught just like face-to-face
courses.
1.97 0.983 Online courses require changes to standard face-
to-face course content.
3.68 1.049 Online courses require changes to standard face-
to-face teaching methods.
4.15 0.838
Note. Likert scale range from 1=Strongly Disagree to 5=Strongly Agree.
Descriptive statistics were calculated for each item and are included in Table 8. The Likert scale for each question ranged from 1=Strongly Disagree to 5=Strongly Agree. Each question was plotted as a histogram to evaluate the shape of the distribution
and a Q-Q Plot was generated to evaluate for normality. All responses included in Table 8 were normally distributed except for “Face-to-face classes provide better opportunities for students to interact than online classes” and “Student and faculty interactions are more effective in face-to-face classes than they are in online classes,” which were highly skewed toward the top of the scale indicating that most respondents feel strongly that interactions between faculty and students as well as between students are more effective in a face-to-face class.
Two pairs of questions within this series were included to explore whether engineering faculty felt there is something different about engineering and non- engineering courses, which ties in to the original premise of the research question. The first two questions, “Online courses in engineering are easier for students than face-to- face courses” and “Online courses in non-engineering topics are easier for students than face-to-face courses”, and the second two questions, “Learning outcomes are comparable in online and face-to-face engineering courses” and “Learning outcomes are comparable in online and face-to-face non-engineering courses”, were asked as pairs to allow a direct comparison of responses. Analysis of the first pair of questions using a paired samples t- test indicated a significant difference in responses; t(263)=-10.431, p<0.001. The comparison of the results from the second pair of questions using the same analysis method also indicated a significant difference in responses; t(260)=-5.061; p<0.001. The results of the comparison of these two pairs of questions indicated that engineering faculty members felt that online courses are easier and outcomes were better for non- engineering courses than for engineering courses. However, the results did not indicate what information these perceptions are based upon or why respondents believed engineering courses were different.
For the rest of the questions in the first section, most of the responses have mean values that are at or near the center of the response range. However, a few results were worth noting as being particularly high or low. The lowest rated response indicated that most respondents disagreed that “Online courses can be taught just like face-to-face courses”; (M=1.97, SD=0.963). This result aligns with another question where respondents strongly agreed that “Online courses require changes to standard face-to-face teaching methods”; (M=4.15, SD=0.838). The two highest rated items were related to interactions, with respondents strongly agreeing that “Student and faculty interactions are more effective in face-to-face classes” (M=4.32, SD=0.819) and that “Face-to-face classes provide better opportunities for students to interact” (M=4.25, SD=0.9.17). The last highly rated result indicated that respondents agree that “Students require more discipline to succeed in online courses”; (M=4.07, SD=0.787).
Table 9
General Perception Items by Undergraduate / Graduate Faculty
Survey Item Undergraduate
Faculty Graduate Faculty
M SD M SD df t(df) P
Online courses can be taught just like face-to-face courses.
1.85 0.847 2.20 1.116 138.9* -2.656 0.009
Engineering design courses can be effectively taught online.
2.24 1.104 2.67 1.152 261 -2.941 0.004
Engineering lab courses can
be effectively taught online. 1.74 0.931 2.07 1.003 260 -2.660 0.008
Note. Likert scale range from 1=Strongly Disagree to 5=Strongly Agree. Levene’s Test for Equality of Variances indicated that equal variances could not be assumed for this item.
Independent samples t-test comparing responses of undergraduate and graduate faculty members with the results of the general perception section of the survey identified several significant differences (p<0.01) as shown in Table 9. The first item in this series
that showed a significant difference between undergraduate and graduate faculty responses related to whether online courses could be taught just like face-to-face courses. However, in this case both undergraduate and graduate faculty indicated that they disagree strongly that online courses can be taught like face-to-face courses. While the overall low rating may be related to the research question, the difference between undergraduate and graduate faculty responses may not.
For the final two items in Table 9, there were differences in undergraduate and graduate level faculty responses concerning whether design courses and engineering labs can be taught online. While the tests showed there was a statistically significant difference, both groups responded negatively toward both of these items, indicating overall disagreement that engineering design courses and labs can be effectively taught online.
The differences in responses between undergraduate and graduate level faculty prompted the inclusion of a question in the interview phase of the study related to exploring these differences. Two predominant themes emerged from the interviews. The first involved a perceived difference in student motivation and maturity between undergraduate and graduate students. The second posited that engineering labs were more prevalent at the undergraduate level and that labs require more interactive and hands-on approaches, which some felt were difficult to deliver online.
Six of the ten interviewees felt that graduate students were more highly motivated and mature learners and that undergraduate students needed more attention, remediation, and interaction. An undergraduate professor with online teaching experience summarized this general opinion, stating:
With graduate students I think you have a more mature and engaged student. A graduate student typically is a self-motivated learner….Whereas, an
undergraduate has to be a little bit more guided and prompted. And sometimes an undergraduate doesn’t know how to learn on their own.
They go on to explain why online courses might be more effective at the graduate level:
If you try to deliver the course in the same way, I can see very easily why a graduate instructor would see that distance learning might be more effective because they are thinking of it in the paradigm of kind of how they traditionally would teach a course….And it gets back to the point where I think we really have to rethink how we do the courses where you can get around these obstacles once you acknowledge there is a difference in learning styles between those two populations.
This insinuates that online courses may inherently require more self-motivation and a certain level of responsibility that may fit a graduate student profile but be lacking at the undergraduate level.
A twist on this theme was raised by a graduate faculty member who had not taught online, noting that a certain level of achievement can be assumed for graduate students, whereas “undergrad seems like the starting point is kind of all over the place. And if the students start asking a lot of basic questions, it could be pretty difficult to deal with online.” Therefore, not only was student motivation and maturity a concern, but undergraduate students may need more interaction and remediation, which some believed to be more difficult to deliver online.
A second theme was noted by several interviewees who mentioned that engineering labs were more prevalent at the undergraduate level, and since labs require more interactive and hands-on approaches, they were more difficult to deliver online.
Therefore, undergraduate instructors may have a more negative perception of online labs. One undergraduate professor who had not taught online stated:
I think the concern about undergraduates is that you have a lot of lab courses and you really have to have hands-on approaches on that. I just don’t see how you teach a lab course like that online, I really don’t see how it can happen.
A graduate professor that also had not taught online compared undergraduate and graduate lab experiences, stating that his “experience in the undergraduate labs is that the students typically need a lot more guidance in the operation of the equipment….I think that graduate students are typically assumed to have most of those skills in hand.” These comments echoed the previous concerns mentioned about labs and the need for interaction.
The comparison via independent samples t-tests of the general perception question items against whether respondents taught at a public or private institution revealed one statistically significant difference. The item related to whether engineering theory could be taught online showed a difference (public institution: M=3.18, SD=1.156; private institution: M=3.73, SD=0.828; t(44.9)=-3.284, p=0.002). However, both means were on the positive or ‘agree’ side of the scale and respondents from private institutions rated the item more positively than those from public institutions. While survey and interview responses do not explain why this might be so, the differences in public and private institution resources, program sizes, etc. may play a role in determining the perceptions of faculty and the implementation of online engineering programs at the individual institutions.
Table 10
General Perception Items by Online Experience Level
Survey Item No Online
Experience Some Online Experience
M SD M SD Df t(df) p
Learning outcomes are comparable in online and face- to-face engineering courses.
2.37 0.996 3.14 1.227 124.3* -4.941 <0.001
Learning outcomes are comparable in online and face- to-face non-engineering courses.
2.69 0.938 3.11 1.074 262 -3.244 0.001
Online courses can be taught
just like face-to-face courses. 1.85 0.848 2.24 1.150 117.5* -2.679 0.008 Online courses require changes
to standard face-to-face course content.
3.80 0.946 3.40 1.218 121.8* 2.624 0.010
Engineering theory courses can
be effectively taught online. 3.01 1.101 3.79 1.027 160.0* -5.566 <0.001 Engineering design courses
can be effectively taught online.
2.18 1.068 2.93 1.178 137.5* -4.880 <0.001
Engineering labs can be
effectively taught online. 1.70 0.880 2.24 1.094 125.3* -3.865 <0.001 Technical /scientific topics can
be effectively taught online. 2.94 1.059 3.74 0.990 263 -5.733 <0.001 Courses heavy in mathematics
can be effectively taught online.
2.83 1.100 3.64 1.046 263 -5.588 <0.001
Non-engineering courses can
be effectively taught online. 3.37 0.912 3.83 0.839 263 -3.839 <0.001
Note. Likert scale range from 1=Strongly Disagree to 5=Strongly Agree. * Levene’s Test for Equality of Variances indicated that equal variances could not be assumed for this item
The comparison via independent samples t-tests of faculty experience with online education versus the responses for the general survey questions identified a number of significant results as shown in Table 10. The first two of the items in this section were related to learning outcomes. In each instance, respondents with no online experience disagreed that learning outcomes are similar in online and face-to-face classes, whether in engineering or non-engineering, and rated engineering courses more negatively than non-
engineering. In contrast, those with at least some online teaching experience tended to agree that learning outcomes are similar for online and face-to-face courses. This is a significant outcome directly related to the research question in that those that have not taught an online course feel negatively that educational outcomes can be achieved online. Therefore, they may be reluctant to teach or implement online courses or online learning methods, especially in engineering subjects.
A question was included in the Phase II interview section to explore why faculty with some online experience felt more strongly that learning outcomes were comparable, while those with no online experience disagreed and said they were not comparable. An undergraduate professor with online experience opined that respondents with no experience may simply be assuming that online courses are not effective. He stated his “suspicion that self-selection – people that really believe that learning outcomes are not comparable are going to choose not to teach any online.”
In contrast, a different theme was shared by three interviewees spanning both the undergraduate and graduate and experience spectra. In summary, they felt that a well- designed online course could have comparable learning outcomes to a face-to-face course. An undergraduate faculty member with some online experience noted that “if you design the class properly, you can have similar outcomes. Probably some of the [survey] response could be biased about how we traditionally put courses together in the past”, adding that “you really have to understand the learning objectives and what you want the students to get out of the course and think about how to design properly.” A graduate professor with no online experience said an online course could be a challenge, “but you can make the learning experience just as effective.” Finally, a graduate engineering professor with online teaching experience stated that “once the criteria was similar … it was hard to tell the difference.”
Additionally, instead of directly discussing learning outcomes, several interviewees provided feedback concerning proxies for learning outcomes, such as one graduate instructor with no online experience who noted that attendance was better in online courses, implying that this might somehow be related to learning outcomes. At least two other respondents noted that resistance to change can have an impact on faculty perception of outcomes. One undergraduate professor with no online experience stated
it is a new thing and… academics maybe more than some other groups, could be resistant to change. You don’t really see the possibilities until you have actually done that, and so, people who have done it have a better feeling about it than people who haven’t.
The survey item related to whether online and face-to-face courses could be taught the same way showed a statistically significant difference between the response groups with both groups strongly disagreeing with the premise of the question. The respondents with no online experience rated the question more negatively, meaning that they felt more strongly that online courses could require changes to their teaching methods.
For the item relating to whether online courses require changes to standard face- to-face course content, survey results showed that both experience groups agreed and those with no online experience felt more strongly that changes are necessary. This result is comparable to the previous question in that respondents with no online experience felt that both their teaching methods and the content would need to change to present a course online, which could be a reason to not pursue an online course.
The analyses of the remaining items in Table 10 showed a significant difference between those with online experience and those without. In each case, respondents with at least some online experience rated the item more highly, i.e. they agreed more strongly
that the topic could be taught online. In some cases, such as engineering theory, both groups felt generally positively about teaching the topic online, while in others such as labs and design courses, both groups felt negatively. In the rest of the topic areas, respondents with no online experience disagreed the topic could be taught online while those with experience felt positively about it. It is also important to note the results of the last item related to teaching non-engineering online. While respondents may or may not have experience teaching non-engineering courses, both groups agreed that non- engineering courses can be effectively taught online (M>3.0). The contrast between this result and results on the labs and design course items could be an important perception related to the gap between the implementation of online courses in engineering and other fields, at least from the perspective of engineering faculty.
Table 11
Effective Delivery of Certain Topics – Descriptive Statistics
Question Mean Standard
Deviation
Engineering theory courses can be effectively taught online. 3.24 1.136 Engineering design courses can be effectively taught online. 2.40 1.152 Engineering labs can be effectively taught online. 1.86 0.979 Technical/scientific topics can be effectively taught online. 3.18 1.100 Courses heavy in mathematics can be effectively taught
online.
3.07 1.144 Non-engineering courses can be effectively taught online. 3.51 0.913
Note. Likert scale range from 1=Strongly Disagree to 5=Strongly Agree.
Descriptive statistics were calculated for each item in the second series of questions in the General Perceptions section as shown in Table 11. These items explored engineering faculty perceptions of whether certain topics important to engineering education as well as non-engineering courses can be effectively taught online. Most responses were generally centered in the response range and are normally distributed,
with respondents rating non-engineering courses higher than any of the engineering topics. The two lowest rated topics were engineering labs (M=1.86, SD=0.979) and engineering design (M=2.40, SD=1.152), which the literature showed are considered key to an effective engineering education.