Section III Evaluation
Chapter 8 The process is the product: Pursuing students’
8.5.1 Collaboration
To understand the implications of using roles supported by DojoIBL in a collaborative inquiry-based learning, more insights about the context and the participants’ perfor- mance were needed. This information was drawn both from 20 hours of video and audio recordings, which captured students and facilitators interactions in the class- room, and from the students’ contributions in DojoIBL. The numbers in table 24 represent the turns taken by each student/role in the different sessions of the inquiry project. On the 24th of November 2016 and 6th of December 2016 the students had a formative and summative assessments during parts of the sessions.
The sessions were always structured as follows; (1) initial introduction, in which the teacher introduces new concepts, present activities, etc. (2) group work, in which stu- dents work together to continue with the project and (3) final wrap up, in which the teacher ends the classroom with reminders and homework for the students. In order to focus only in the collaborative part, in table 24 only the group work and the verbal
communication is represented. Interestingly, towards the end of the inquiry project an increase in the number of interactions can be observed. Around 80% of the turns taken took place in the last two weeks of the inquiry.
Table 24: Students’ turns taken during the inquiry project.
Case 17 N ov 22 N ov 23 N ov Six b lin d m en 24 N ov VR ac t. 6 D ec 7 D ec A rg um en t 8 D ec 13 D ec Total Teacher 4 11 6 0 8 25 28 20 102 Designer 5 7 8 4 13 40 38 30 145 Organizer 2 0 1 0 8 1 4 3 19 - 0 5 7 0 11 0 1 0 24
Checks the work 15 0 0 0 26 41 15 30 127
Research 2 5 8 0 14 16 33 23 101
Keeps all the work 1 3 1 0 0* 2 10 27 44
Cheering up 1 0 1 0 0* 0 8 25 35
Writing 1 0 2 0 0* 0 4 21 28
Checks if everything is fine 1 0 5 0 0* 1 4 8 19
Research 0 0 1 0 0* 0 1 0 2
* Technical problem with the recordings
In order to deepen in the analysis of the collaborative aspect, a discourse analysis was carried out for the three units of analysis; the classroom discourse, the DojoIBL chat logs and DojoIBL contributions. The analysis was done using the CoI framework coding scheme (see Appendix D). It consisted of four main components; cognitive, social, teaching and metacognitive presence, which in turn had several categories un- derneath. The following excerpts are examples of how one student or teacher interac- tion can be coded under different categories. In the first example, the teacher interac- tion has been coded as Encouraging, … reinforcing student contribution, from teaching presence, and Procedural planning, from metacognitive presence.
Teacher: This is beautiful. I love the way you work. Now, you have 15 minutes left. Keep it up! Whatever time you waste now you will have to catch up in the weekend.
The second example has been coded as Judgement and Triggering Event from the cognitive presence category. In this case, the student besides judging the actual design opens the floor for discussing about it.
Student: Please go to the introduction in the slides. Shall we keep that pic- ture there or shall we take it off? I don’t like that picture.
The third example has been coded under Request information (applying strategies) and
Asking questions, from metacognitive and social presence, respectively. Firstly, the
student is asking a question in order to get some support and then requests infor- mation from another student.
Student: Can you do strand 2, student E? Which one are you doing student F?
Last example covers the summary of a discussion (from teaching presence) in which both students agreed on who will take the lead of the design of the presentation ex-
pressing agreement from (social presence)
Student: I have talked to him (pointing to Student B). We talked about it, because Student B is good at designing. And he will do the design.
Given the examples above and applying the same procedure to all the participants’ interactions, table 25 showed that first, the cognitive presence had an important role achieving the highest numbers in two (classroom discourse and contributions in Do- joIBL) of the three units of analysis. The contributions for cognitive presence in Do- joIBL are higher than the chat contributions. This aligns with the literature since commenting in DojoIBL has been preferable to chat messaging when the goal was to achieve high-order cognitive learning (Garrison et al., 2001). Second, the teaching presence in the classroom discourse was similar for the students and the teacher. This reveals an important role of the students in the teaching presence during the inquiry project while sustaining the social and the cognitive presence and the community of inquiry. Third, the social presence, through its interactive response category, suggests that express a willingness to maintain and keep up the interpersonal interactions (Rourke, Anderson, Garrison & Archer, 2007) is essential to any learning community. Last, the bottom section shows that students, during the classroom, reflected in action (regulation of cognition) about their metacognitive processes while the comments in DojoIBL have been mainly used to reflect on action (monitoring of cognition). In both cases a higher number was expected.
Table 25: Students’ turns per CoI presences in the different channel and context of the inquiry project.
Process Classroom discourse DojoIBL Chat discourse DojoIBL contributions Student Teacher Student Teacher Student Teacher
Social presence 233 45 43 7 27 3 Affective response 19 0 8 0 3 0 Cohesive response 32 2 12 7 7 0 Interactive response 182 43 23 0 17 3 Cognitive presence 260 20 28 3 178 0 Exploration 141 7 24 3 86 0 Analysis 53 8 1 0 40 0 Integration 66 5 3 0 52 0 Teaching presence 83 81 7 11 2 6 Direct instruction 59 52 6 6 0 3 Facilitating discourse 24 29 1 5 2 3 Metacognitive presence 120 40 14 0 24 0 Knowledge of cognition 7 0 0 0 0 0 Monitoring of cognition 23 19 3 0 24 0 Regulation of cognition 90 21 11 0 0 0 8.5.2 Roles
To deepen on the understanding of how roles worked during the inquiry project, this section presents an analysis of how the different roles were carried out taking into account the four dimensions of the CoI framework (social, teaching, cognitive and metacognitive). Graph 4 shows to what extent each role contributed to the different CoI dimensions. It can be seen that there is a pattern that repeats for most of the roles. Besides the ‘organizer’ and ‘cheering up’ roles, the cognitive dimension is followed by the social, metacognitive and the teaching dimension.
Additionally, it can also be seen that the two roles, ‘checks the work’ and ‘designer’, involved in the dispute on the ‘analysis’ phase, achieved the highest values compared to the rest of roles. This finding deserves a thorough look. First, the dispute began from a conflict of interest between two roles that can be seen in the following ex- cerpts:
- Designer: I am making the presentation and he is the one checking the presenta-
tion…
- Checks the work: Then what do I check now?
Both roles were entitled to have a word in the issue, and they defended their right to do so mentioning their responsibilities. This means that roles created interdependency among learners as their roles overlapped in one of the responsibilities. This overlap and the interdependency generated a high number of interactions (in all the dimen- sions, see graph 1). However, such overlap between roles has not been seen in other situations during the inquiry project. Opposite to what the intention of roles is, they
did not influence the behavior of the learner (in greater or lesser degree), because they were performing similarly based on the CoI dimensions. This finding contrasts with the reflections made by the learners in DojoIBL (see Appendix C) in which 7 out of 8 students mentioned that roles work as prospected.
Graph 4: Role's contributions to the different CoI dimensions.