We have developed MOOCchat and an instructional method to form syn- chronous discussion groups in MOOCs using the tool. Also, the software tool and the collaborative task design were tested out in an actual online course. It was the first step in understanding how software can guide students collabora- tion in an online education environment, but much work remains to be done.
The first takeaway in terms of the software design is that the staged model works very well in structuring students behavior in using the tool. It seems that the students perceived the linear staged model straightforward, judging from no attrition observed out of 61 students who began the task. None of the open- ended feedback comments in the survey had negative report on the difficulty in using the tool or following the collaborative task. More encouragingly, the staged model worked well in guiding the cognitive tasks (e.g., reading passages and figuring out the answers) as well as the behavioral tasks (e.g., clicking but- ton to proceed and typing texts for discussion). This could be useful design implication for future works.
As the survey feedback shows, MOOCchat successfully facilitated students interaction around the task. The task design wise, it seems that the separa- tion of individual task from collaborative part (discussion) and deliberately us- ing the outcomes of individual task (answers) in the collaborative task (sharing the group members’ answers) contributed to the feasibility of performing the task, judging from the responses to the survey question “Discussion was help- ful for final choice”. For further investigation, we conducted another iteration of MOOCchat in a crowdsourcing platform to verify the quantifiable effectiveness of this approach. In that experiment, participants who were grouped for the col- laborative tasks marked higher rates of correct answers in the quiz, compared to solo participants; among the 169 responses provided by solo participants, 84 (50%) were correct, where among the 269 responses from crowd workers who participated in a discussion group (dyads or triads), 169 (63%) were cor- rect. This difference turned out to be statistically significant (Fisher’s two-tailed exact test, p ¡ 0.01). This supports that MOOCchat approach successfully facili- tated the students interaction, which played a key role in performing the tasks
[31].
This study also had several limitations, which opens the possibility for the next step going forward. The online learning environment renders a lot of po- tential for novel approaches to learning with lower cost and higher accessibility, however, it also rendered serious defects as a research testbed. For example, the synchronous online collaboration is heavily affected by disperse attendance at any given time because online learning environment heavily relies on an indi- vidual’s self-directed or self-paced learning, which is on the opposite of struc- tured learning. It might be possible to strictly schedule the participation to alle- viate this issue, but this scenario less reflects on the real-world learning context as a natural field test. Consequently, the MOOCchat experiment left almost three quarters of the overall voluntary participants ungrouped for the collabo- rative task. This can’t be the ideal setting for the research trying to develop and test collaborative approaches to learning with students in the actual learning context.
We argued that MOOCchat approach effectively facilitated interaction. However, given that the students could answer multiple choice questions and collaborate to identify and correct errors in those answers through text chat, there is no evidence that the interaction has brought about learning; a possible alternative explanation is that the students had a chance to give a second look into their answers when others in the discussion group express different opin- ions, or even more simply, they could have corrected the answers just following others’ opinions. MOOCchat approach could be justified for its own purpose that the review and self-evaluation through peer discussion on the coursework materials were much needed to help the specific target students learn. How-
ever, the findings are hardly sufficient to expand to social learning in introduc- tory programming practice due to the use of the ad-hoc grouping and chat tool. Possible reasons include the simplicity of the task (answering multiple choice questions), the way of collaboration (a free-form text chat for a limited period of time), the level of engagement (self-directed online learning not being graded). This calls for further investigation in students tasks taking longer term, with higher complexity, which better engages them in and in better structured ways. Another limitation, considering the general goal of this dissertation aiming to provide a social learning method for collaborative programming, was that the collaborative quiz task we designed didn’t provide the authentic experience in programming or yield potentials to iterate in that direction. Given the course materials that addressed programming issues and practice (Ruby on Rails), giv- ing Ruby-related questions for collaborative quiz sessions would have been as close as MOOCchat approach can get to programming education through so- cial interaction. Even with those questions, it would have been more of another replication of syntax-oriented learning in programming with extra communica- tion channels among peers, which doesn’t fulfill this dissertation’s vision that introductory programming education should be designed toward collaboration.