1
Redesigning Online Deductive Logic to Improve Retention
Daniel R. Boisvert
Marvin J. Croy
UNC Charlotte SoTL Project Report DanBoisvert@uncc.edumjcroy@uncc.edu
Abstract
Over the past three years, the Department of Philosophy at UNC Charlotte has developed and implemented a fully online section of Deductive Logic. The central problem for these sections thus far has been its low retention rate, defined as its 'ABC' rate, which is half that of comparable face-to-face sections of Deductive Logic. After a brief review of the literature concerning the causes of low retention in online courses, we hypothesized four factors contributing to our own low retention rate: inadequate student-to-student and student-to-faculty interaction; unintuitive course web site design that also failed to adequately contextualize the course material; little sense of community; and unrealistic student expectations regarding the difficulty of online learning. We have since created and implemented a variety of course design and web site modifications, e.g. required group activities, mandatory live sessions, audio and video elements, orientation to online learning, and others. Using course web site report data, student grades, and redesigned course evaluations, we aimed to determine whether these modifications have led to increased interaction, more intuitive course navigation and conceptualization, a deeper sense of community, more realistic student expectations of what is required for successful online learning, and ultimately, higher retention. The modifications that appear to have been most helpful were those intended to improve student-faculty communication and the course web site. The modifications that appear to have been least helpful were those intended to improve student-student interaction, sense of community, and student expectations. While still low, overall retention has improved thirty-three percent.
1. Introduction
Retention rates for online courses are consistently lower than those for face-to-face courses (Simpson 2004; Wojciechowski and Palmer 2005). No exception is the retention rate for the online sections of deductive logic that we teach at the University of North Carolina Charlotte, which was less than half that of comparable face-to-face sections of the course. This retention rate is especially problematic, since this course is required of our Philosophy majors and is one of only several options that satisfies a Mathematical Reasoning requirement in our university's General Education program.
After reviewing the literature concerning the causes of low retention in online courses and
participating in the five-week workshop "Planning Your Online Course," offered by our University's Center for Teaching and Learning, we hypothesized four contributing factors leading to our low retention rate: inadequate student-to-student and student-to-faculty interaction; unintuitive course web site design that also failed to adequately contextualize the course material; little sense of community; and unrealistic student expectations regarding the level of commitment required for successful online learning in a college course. Accordingly, we created and implemented numerous course design and web site modifications, including required team discussion activities, mandatory and optional
synchronous workshops, multi-media resources, orientation to the nature of online learning (and our course), and others.
After implementing these modifications, we aimed to determine whether these modifications led to (i) increased student-student and student-teacher interaction, (ii) more intuitive course navigation and conceptualization, (iii) deeper sense of community, (iv) more realistic student expectations of what is required for successful online learning, and, ultimately, (v) higher retention. To do so, we used report data from our University's learning management system (Moodle), student grades, and student course evaluations adapted from the Community of Inquiry rubric.
2
2. Method, Materials, and Procedure
2.1 Subjects
One 100%-online section of PHIL 2105 Deductive Logic was offered at UNC Charlotte during each of the Fall 2010 (60 students), Spring 2011 (60 students), Summer 2011 (41 students), Fall 2011 (58 students), and Spring 2012 (59 students) semesters, for a total of five such sections. We collected and analyzed different types of data, described below, for all students registered as of census date for each of these five sections, for a total of 278 subjects from these sections. In addition, and for comparison with face-to-face courses, retention data only was collected from one face-to-face section of the same course during Fall 2010 (40 students). No demographic information was collected on any of the 318 total subjects. (Based on forum comments alone, instructors perceived no noticeable demographic difference between the online only students and either the face-to-face students or students from general UNC Charlotte population.)
2.2 Instructors
The same instructor, Instructor A, taught all sections of the course that were part of this study, including the face-to-face section, with the exception of the Spring 2012 semester, which was taught by Instructor B. Instructor A is full-time lecturer with more than fifteen years experience teaching deductive logic. Instructor B is a part-time lecturer with six years experience teaching deductive logic.
2.3 Student Work and Grades
With three exceptions, students in every section of the course, including the face-to-face section, were graded in exactly the same way on exactly the same online assignments. Two exceptions are the Fall 2010 online and face-to-face students, who were not required to, and so did not receive a grade for, participating in online (or face-to-face) discussion. The other exception is the Spring 2011 students, who were required to "attend" two required synchronous online meetings, which constituted a portion of each student's overall participation grade.
2.4 Implementation of Enhancements
Course enhancements, described in Table 1, were implemented beginning Spring 2011.
2.4 Data
Using a combination of report data from our university's learning management system (Moodle), student grades, and student course evaluations adapted from the Community of Inquiry rubric, we intended to evaluate the following:
(i) quantity and quality of student-student and student-teacher interaction;
(ii) ease of course navigation and effective contextualization of content and activities; (iii) students' sense of community or belonging;
(iv) clarity of expectations, both of the requirements for successful online learning and of our course objectives; and, of course,
(v) retention rate, defined as the percentage of students registered as of census date who received a grade of 'C' or higher. (Our university does not use plus or minus grades.) The relationships among these areas of emphasis, the specific enhancement strategies, and their evaluation methods are described in Table 1.
3. Results and Discussion
3.1 Quality and Quantity of Student-Student and Student-Teacher Interaction
We introduced several kinds of multi-media enhancements to improve the quality and quantity of student-teacher interaction. With respect to quantity, few students interacted with these very much:
3
Table 1. Areas of Emphasis, Enhancement Strategies, and Evaluation Instruments Areas of Emphasis Enhancements/Intervention Strategies Evaluation Instruments Quantity and quality of
student-student and student-teacher interaction
Required team discussion activities with instructor feedback
Mandatory and optional live group Wimba sessions with instructor
Moodle report data specifying
percentage of students who listened to the audio welcome message, viewed video tutorials, attended Wimba sessions, etc.
Student grades specifying percentage of students receiving at least a 'C' for their discussion grade (which accounts for both quantity and quality of
discussion activity).
Course evaluation questions (e.g. 'Instructor actions reinforced the development of a sense of community among course participants')
Intuitive course navigation and contextualization
Weekly units
Clearly defined objectives, assignments, and activities for the course and for each learning unit Overview and audio welcome
message for each learning unit that explains how the material for that unit relates to the overall course goals and to the other material for other units
Clear sidebar navigation for one click access to all important information about the course (instructor information, course goals, expectations, quizzes, forums, Moodle navigation handouts, etc.)
One click access to weekly units
Course evaluation questions, such as 'The course web site was easy to navigate' and 'The weekly overviews helped me better understand the purpose of the weekly activities and how they were related to the overall purpose and progression of the course'.
Deeper sense of community
Required team discussion activities Mandatory and optional live group
Wimba sessions
Audio welcome messages for each weekly unit
Instructor video "think out loud" tutorials
Instructor "encouragement emails" sent immediately upon a student's failure to complete a weekly assignment.
Course evaluation questions, especially many questions evaluating for "Social Presence" (e.g. 'Getting to know other course participants gave me a sense of belonging in the course', 'I was able to form distinct impressions of some course participants') and some for "Teaching Presence" (e.g. 'Instructor actions reinforced the development of a sense of community among course participants.')
More realistic expectations about the nature of and requirements for successful online learning
One-week Orientation unit Moodle report data specifying
percentage of students who viewed the orientation resources
Student grades specifying percentage of students who scored at least a 'C' on an orientation quiz.
Course evaluation questions (e.g. 'By end of orientation, I had a clear sense of the time commitment often required to do well in the course')
4
Weekly audio messages were created and uploaded each week during the Spring 2011 semester. These provided an overview of the week's coming work and summarized some important announcements for the coming week: 23% (14/60) of students interacted with these. (Because so few students listened to these, and because their creation was labor intensive, they were dropped from future sections of the course.)
Several video tutorials were created and assigned in which the professor led students through each step of a number of logic exercises, including the professors thoughts on the how's and why's of each step: 29% (46/158) of students interacted with these.
"Optional" video resources were introduced Fall 2011: 14% (16/117) of students interacted with these.
Two live, synchronous, online sessions were required of students during the Spring, 2011 semester, and grades were given for attendance: 53% (32/60) attended these sessions. We also introduced mandatory Team discussion activities, worth ten percent of a student's overall course grade, to improve the quantity and quality of student-student interaction. We graded students according to the quantity and quality of their posts. Only 29% (64/218) received a grade of 'C' or higher for this activity.
We also designed a course evaluation containing a number of questions from the Community of Inquiry rubric to evaluate students' perceptions of the quality of instructor-student and student-student interaction. Thus far, only students in the Spring 2012 section have taken the survey, and the sample of twenty-three students who responded is relatively small.
Students evaluated one-way communication from the instructor very highly. For example, 85% agreed or strongly agreed that 'The instructor clearly communicated important course topics' and that 'The instructor clearly communicated important course goals'.
Students evaluated instructor-student interaction as mixed. For example, while fewer than 65% agreed or strongly agreed that the 'Instructor helped to keep course participants
engaged and participating in productive dialog' and that 'The instructor helped to keep course participants on task in a way that helped me to learn', upwards of 86% agreed or strongly agreed that 'The instructor helped to focus discussion on relevant issues in a way that helped me to learn' and that 'The instructor provided feedback in a timely fashion'.
Students evaluated the quantity and quality of student-student interaction especially low. See below the section Deeper Sense of Community.
3.2 Intuitive Course Navigation and Contextualization
We devoted a lot of energy to redesigning the course web site to make it easier to navigate, more intuitively understand activities and due dates, and better contextulize what students were doing and why. We also redesigned the course evaluation instrument to evaluate students' perceptions of the web site, and the results were positive across the board.
91% of students agreed or strongly agreed that 'the instructor clearly communicated course topics'
87% agreed or strongly agreed that 'the instructor clearly communicated important course goals'; 87% agreed or strongly agreed that 'The instructor provided clear instructions on how to participate in course learning activities'; and 83% agreed or strongly agreed that 'The instructor clearly communicated important due dates/time frames for learning activities'. 87% of students agreed or strongly agreed that 'the course web site was easy to navigate'. 87% of students agreed or strongly agreed that the weekly overviews 'helped me better
understand the purpose of the weekly activities and how they were related to the overall purpose and progression of the course'.
One student volunteered, "Everything was very well structured and stated for easy navigation and completion of assignments."
3.3 Deeper Sense of Community
In redesigning the course, our primary aim was to develop among a students a much deeper sense of community and belonging. For example, we created mandatory weekly discussion activities, audio
5
welcome messages for each week of the course, instructor "how to" videos, and other enhancements. The results are disappointing across the board.
Fewer than 35% agreed that the 'Instructor's actions reinforced the development of a sense of community among course participants'.
Fewer than 28% agreed that 'Getting to know other course participants gave me a sense of belonging in the course'.
Fewer than 40% agreed that they were "able to form distinct impressions of some course participants'.
Fewer than 32% agreed that 'Online or web-based communications is an excellent medium for social interaction'.
Fewer than 40% agreed that 'I felt comfortable participating in the course discussions'. Fewer than 47% agreed that 'I felt comfortable disagreeing with other course participants
while still maintaining a sense of trust'.
Fewer than 48% agreed that 'I felt that my point of view was acknowledged by other course participants'
Fewer than 48% agreed that 'Online discussions were valuable in helping me appreciate different perspectives'.
Students volunteered: "If the participation portion of the web site were revamped in some way, I feel like the course could be better overall"; and "I don't feel that the participation forums are living up to all that the PHIL dept. is expecting from them. They really are not very helpful, especially for those that are having trouble with the work. Rather, I feel that it woudl be much more helpful if Professors made videos to teach each part of the course, as if the online students were apart of a regular class."
3.4 More Realistic Expectations for Successful Online Learning
To better orient students to the nature of online learning and what it takes to succeed in an online environment, we created a one week Orientation unit, introducing students to online learning and to Moodle, our learning management system. The Orientation consisted of a number of resources, including handouts, links to web sites, and a summary quiz. Students interacted with some of these resources much more than others.
Students interacted most with the following resources:
Orientation quiz (beginning Fall 2011): 90% (105/117) 'Managing Discussion Forum and Email': 51% (80/158) Students interacted much less with the following:
'Are Online Courses for Me?': 19% (30/158) 'Myths About Taking Online Courses': 16% (25/158) 'Tips for Successful Online Learning': 3% (4/158) 'How to Navigate Moodle': 6% (10/158)
'Moodle Orientation for Students': 6% (10/158)
'Making Moodle More Efficient and Less Cluttered': 13% (21/158)
3.4 Retention
All of the modifications were of course ultimately desgined to improve retention. While overall
retention remains low, it has improved. Fall 2010 was the most recent "pre-modified" online section of the course, and of sixty students in this section, we retained eighteen, a 30% retention rate. Since the modifications, we have retained 40%, a thirty-three percent improvement. For comparison, during Fall 2010's face-to-face section of the course, we retained 28 of 40 students, for a 70% retention rate. The more specific numbers are presented in Table 2.
6
Table 2. Retention Rates and Improvement by Semester and Overall
Semester Retention (students earning
at least a 'C') Improvement Fall 2010 Face-to-Face 70% (28/40) Fall 2010 Online 30% (18/60) Spring 2011 37% (22/60) 23% Summer 2011 44% (18/41) 47% Fall 2011 38% (22/58) 27% Spring 2012 42% (25/59 40% Overall (Post-Intervention) 40% (87/218) 33% 4. Conclusion
Overall, the results are moderately encouraging. For although retention in PHIL 2105 Deductive Logic since Spring 2011 remains a low 40%, and so remains 43% lower than face-to-face sections of the course, that number does constitute a 33% improvement.
The modifications that appear not to have succeeded are those for improving student-student interaction, sense of community, and student expectations. These constitute the most disturbing aspect of the results: that very few students feel they are part of a helpful, interactive community of inquirers.
The modifications that appear to have succeeded are those for improving student-faculty communication and the course web site. These constitute the most encouraging aspects of the results: that most students find it easy to know what their objectives are, what activities they need to perform, and when those activities are due.
References
Simpson, O. (2004). The impact on retention of interventions to support distance learning students. Open Learning, 19(1), 79-95.
Wojciechowski, A., and Palmer, L. B. (2005). Individual student characteristics: Can any be predictors of success in online classes? Online Journal of Distance Learning Administration, 8(2).
