3.4 Data analysis
3.4.2 Data analysis in current study
3.4.2.2.1 Coding of speech
This section draws heavily on the discussion of student learning set out in Section 2.6, and Figures 2-6 and 2-7. Those suggest that speech can be broken down into a number of categories to capture different parts of the meaning-making process. In this research, speech was coded into seven major categories derived from Hmelo-Silver’s research design (Chernobilsky et al., 2005b; Hmelo-Silver et al., 2008). Hmelo-
Silver’s approach was supplemented by adding a concept of internalisation (to capture who was leading the meaning making between the students and the facilitator) and of scaffolding (Hill & Hannafin, 2001; Stålbrandt, 2007) to explore the process of learning. Each verbal utterance was coded to at least one of these categories in the transcription and analysis process. The categories were developed from the literature on multimodality discussed in Chapter Two:
1) Content of the talk; 2) Collaboration;
3) Responses of ideas/complexity; 4) Knowledge;
5) Metacognition; and 6) Interpretation.
As above the work of Hmelo-Silver et al. (2008) is important to the development of these concepts and the concepts listed are drawn from their study (see Figure 2-6 and related discussion in Section 2.6). However, these concepts are supplemented by ‘internalisation’ (Table 3-14), developed by the researcher in the early stages of the data analysis. In addition, the important issue of scaffolding in a PBL context (see Section 2.2) is developed from the work of Stålbrandt (2007) as it captures a critical aspect of the PBL pedagogic model.
A given stretch of speech could be coded to any number of these categories, from one to all seven, allowing for a detailed analysis of the flow, content and meaning of each block of social interaction. In turn, each of these categories was broken down into second or third order categories to ensure a detailed analysis was carried out on a consistent basis (so, for example, ‘Questioning’ is divided as generating either a ‘short’ or ‘long’ answer and in turn, short answers are split between categories such as simple verification or whether more information was provided). Each category was given a simple code and this was used in the tables (see List of Abbreviations and Appendix 6).1
1
Some abbreviations do not follow the general principle of using the initial letter of the main terms in the full form, e.g. Task-related Talk = TT, while Tool-related Talk = TR since otherwise there would be two abbreviations TT. In other cases this benign inconsistency occurs because sometimes the initial letter of the abbreviation is taken from the main category and the second letter from the sub-category or
Content of the talk (Chernobilsky et al., 2005b; Hmelo-Silver et al., 2008) was broken down into four categories, depending on whether they were discussing the task, the tools needed to complete that task, the concepts behind the task or the talk had ceased to be task-specific, as shown in Table 3-8
.
Table 3-8: Coding of content of talk
Category Sub-Categories Example
Content of Talk Task-related Talk (TT) Keep to those specifications guys
Tool-related Talk (TR) you need to know what your calculator is
doing.
Concept-related Talk (CT) you can use the resolution of forces too
Personal Talk (PT) But it’s okay, this semester is all right. Keeping
up.
The coding of collaboration required a more complex scheme to capture the full range of possible interactions (Chernobilsky et al., 2005b; Hmelo-Silver et al., 2008b). This included identifying the introduction of new ideas and their modification, agreement and disagreement, summarising the discussion, forms of collaboration by the facilitator and various modes of questioning. The full list is given in Table 3-9.
Table 3-9: Coding structure for collaboration
Category Sub-Categories (1) Sub-Categories (2) Example
New ideas (N) subtract the down…
Modifications of ideas (ModI)
for the up and down you’d go –
Agreements Task-related
Agreement (TA)
The square, yeah Conceptual
Agreement (CA)
Yeah that’s right. Disagreements /
conflicts
Task-related Disagreement (TD)
No, because this one is going down this way
Conceptual Disagreement (CD)
10… 8.61 – no that’s not it… Acknowledgement
of others’
contribution (Ack)
you got it gosh…
Facilitator’s input Monitoring (FM) Guys, you need to finish this
work on your own time Explaining Tool-
related utterances (FT)
I recommend everybody to buy the (Hugo) Statics book
Explanations (FE) that become a negative Y and
then the Xs…
Questioning Short Answer Verification (SV) Yes, I got it right for once!
Quantification (Quant)
there’s a number 275 paddle-pop sticks
Long Answer Interpretation (LI) What did you end up
Task oriented and Group Dynamics
(GD)
Or you would get 8,669… Self-Directed
Learning (SDL)
this is Y going up and this is Y going down,
Need Clarification (NC)
Was this done in the lecture … Self-answered
Questions (SQ)
So then the difference between that and that, -5 for the Y, Questioning
Facilitators (QF)
What if I put it up like that?
Thus collaboration can range from the introduction of new ideas to modification of ideas, statements of agreement or disagreement, questioning and the scaffolding of concepts and ideas in the meaning-making process.
The process of responses and idea development (Chernobilsky et al., 2005b; Hmelo-Silver et al., 2008) was designed to capture the nature of the interaction as shown in Table 3-10.
Table 3-10: Coding of responses
Category Sub-Category Examples
Agreement with Facilitator (AF) Student D (replies to the
facilitator): Yeah
Agreement with PBL Member (AM) Yeah week 8 at the moment
Seeking Clarification (Clarif) How long does the uni year go for?
Brief Answer (BA) So we’ve got another six months
Elaborated Telling (ET) Then we have another one on top
and
Elaborated Explanation (EE) And that’s the sort of one that you
want to plan for.
Justification of ideas Personal Experience
(PersE)
It will, but this will probably still be stronger.
In turn, knowledge (Chernobilsky et al., 2005b; Hmelo-Silver et al., 2008) was coded to indicate the basis for a statement of belief. In Hmelo-Silver’s model this is described as ‘justification’ and coded as shown in Table 3-11.
Table 3-11: Coding of knowledge
Category Sub-Category Example
Knowledge Conceptual Knowledge (CK) Then I’ll have to figure out the extraction
Prior Experience (PE) I thought the X one was down because
Metacognition is drawn from the concept of scaffolding (Hill & Hannafin, 2001; Stålbrandt, 2007), which relied on an analysis of the framework being adopted to justify a given statement, as shown in Table 3-12.
Table 3-12: Coding of metacognition
Category Sub-Category Example
Monitoring Individual Monitoring (IM)
(checking on personal progress)
: so you need to know your sine and cosine rules because in terms of the parallelograms that set up the forces, Group Monitoring (GM) (checking
on group progress)
Listen guys, if I just uh, have your attention please,
Self-Directed Learning (MSDL) Then I found the S from that.
Planning Theory-driven Planning (ThP) So there are two ways of solving it
Unjustified Planning (UP) You’ve got to do a drawing…
Interpretation involved a judgement about the intensity of a given interaction and the extent to which it was either simple or complex, as shown in Table 3-13.
Table 3-13: Coding of interpretation
Category Example
Low-level (IL) Write them down
Internalisation was coded to indicate who was directing the meaning making and learning being observed. In this case, the coding scheme in Table 3-14 was developed specifically for this research and provides a shorthand overview of the interaction between the students and the facilitator.
Table 3-14: Coding of internalisation
Category Example
Peer-to-Peer Internalisation (IPP) Student B: Yeah, yeah, yeah right… Yeah,.
Facilitator-to-Student Internalisation (IFS)
Facilitator: The only way you’re going to get a strong sort of bond,
Student-to-Facilitator Internalisation (ISF)
Student A (to the facilitator): And then the next one is going to overlap them
Table 3-14 is not based on the literature review in Section 2.6; instead it was developed as part of the coding to indicate when the facilitator and when the students are leading the meaning-making process. This category was developed to allow an understanding of the extent of interaction within the student group and between the students and the facilitator. In practice the nature of such interactions could also studied by exploring the flow of each discussion (in other words when the active meaning making shifted from supervisor to students and vice versa) and thjs meant that less reliance was placed on internalisation in practice when completing the analysis than appeared to be the case when the conceptual categories were originally constructed.
Scaffolding (Hill & Hannafin, 2001; Stålbrandt, 2007) is an important part of PBL learning and has been coded in these transcripts. The coding scheme below has been adapted from the work of Stålbrandt (2007), as shown in Table 3-15.
Table 3-15: Coding of scaffolding
Category Example
Conceptual Scaffolding (CS) that I’ve seen in my experience have been what I call box-shapes
Metacognitive Scaffolding (MS) in order to get the result in force you need to know the angle too
Procedural Scaffolding (PS) 8.6 squared plus 7.4 [background noise] and the square root of all
these
Strategic Scaffolding (SS) so you could actually stand on a wire frame structure.
Technical Scaffolding (TS) you need to know your sine and cosine rules because in terms of the
parallelograms that set up the forces,
Scaffolding is an important part of how the facilitator seeks to guide student meaning making within PBL. At its core, it reflects the intention to allow students the opportunity to structure their own thinking about the task rather than offer the final
framework (conceptual, metacognitive and strategic) or providing more task and subject focussed information (procedural and technical).