Methodological reflections

This section offers some brief reflections on my use of the tutorial programme as a vehicle for studying learning, on IQA as I applied it, and on my role and involvement in both.

As described in section 4.4.1, the structure of the tutorial programme allowed me to use cooperative learning and small-group discussion, which were not feasible in the mainstream group. This approach demanded intense involvement from me in preparation for as well as facilitation during the sessions. I found teaching in this way rewarding and energising, but knew that the programme was very much a one-off experience that could not feasibly be replicated for the class of a few hundred students within current budget and timetable constraints.

In offering the tutorials, I was constrained by the mainstream course arrangements alongside which the tutorials ran. BSTS 201 is offered to a few hundred students on two campuses, with common content and assessments, several lecturers and a course coordinator. I did not have leeway to depart from the pre-existing syllabus or to redesign assessments to align with the pedagogical approach in the tutorials.

95

Assessment relies heavily on multiple-choice questions, and there is little room to redesign these in line with a threshold concepts orientation. This may have meant that the potential impact of the threshold concepts-enriched tutorial programme was not fully realised, as participants’ learning continued to be driven largely by prevailing assessment approaches (Biggs, 1996). Concerns about assessment arose in individual reflections during the group and reflective writing, and are discussed in later chapters.

I was fortunate to have access to most of the tutorial exercises I used in a suitable format (Triola, 2015; Weiss, 2017), and was able to map appropriate activities to what I perceived to be the typically troublesome, transformative elements of the BSTS 201 curriculum, using only minor adjustments such as the conversion of metric measurements in the examples to the metric system used in this country. Being able to use these activities, which were closely aligned with the threshold concepts theory, not only facilitated the development of the programme but also increased my confidence in the approach. This was borne out by the participants’ responses. The ways in which concepts were presented in the tutorial exercises allowed for a different take on the mainstream lectured material, as well as offering the chance to complete or resolve partial or inaccurate prior understandings.

I had to revise my initial plan to use purposive sampling to select participants from volunteers from the class. As it happened, the number of volunteers was only slightly higher than my intended sample, so I invited all of them to take part in the programme. In effect, the participants therefore self-selected. Over the course of the semester, a few stopped attending tutorials, compounding the element of self- selection of those who remained to take part in the IQA processes. While the lack of attention to representivity is not a major concern since the intention is not statistical generalisability, I am aware that there may have been different insights into learning in statistics from those who did not select themselves into the tutorial programme, or who opted out before the end, which will remain uncaptured.

I planned to have the content covered in the tutorial activities keep pace with the content taught in the mainstream lectures. However, participants that were from the other half of the class taught by another lecturer were sometimes behind in the sections covered in the tutorial programme. As such, I had to repeat the tutorial activity the following tutorial session, so that the other participants could benefit from doing an activity based on content that they have already been acquainted with during the lecture periods. Nonetheless, those students who were already familiar with the content were given the

96

opportunity to play the role of tutor and help the other students to understand the new content. (This aspect of learning is discussed further in later chapters). As such, the repetition of a tutorial activity along with other external (out of my control) factors42 _{placed constraints on the number of tutorial} periods that I was able to hold over the semester, and limited the span of content that I had planned to cover during the duration of the tutorial programme. Thus, the tutorial activities that I designed for the discussion of the statistical topics of hypothesis testing and Chi-squared tests, were not attempted in the tutorial sessions. As a result, insight into learning these topics will remain uncaptured.

My use of IQA revealed some limitations and possible hazards, as well as the advantages noted in describing the approach in section 4.7. Many of my concerns relate to the pivotal role of the focus group, on which the rest of the approach may be seen to depend. First, the communal nature of the focus group, and its emphasis on group reality, seem to reflect an assumption that there exists a single, albeit complex, group reality that can be satisfactorily captured in the SID. The IQA guidelines (Northcutt & McCoy, 2004) do not offer steps to follow if participants cannot reach complete agreement before time and energy run out. In the focus groups, my observations suggested that the participants reached consensus relatively smoothly, and I did not have to respond to this type of dilemma. A concern remains, however, that in seeking consensus, the focus group processes might simply be masking conflict and attaining compromise — the appearance of agreement. The silent nature of brainstorming and clustering in IQA, while offering advantages, may not reveal unresolved conflicts of meaning among participants.

The interviews may address some of the concerns around consensus and silence, by offering an opportunity to explore individual variations in the meaning of affinities. However, the interviews are themselves bounded and structured by the affinities, which inform the interview protocol and therefore largely determine the issues to be discussed. Thus, while the grounded, participant-driven nature of affinities (and their interrelationships as depicted in the SID) is one of IQA’s key strengths, this may be a double-edged sword: once confirmed, the affinities are set, and the researcher and readers are called on to trust that they are a fair representation of the participants’ reality. While Northcutt and McCoy (2004) point out that the interviews can act as a check of the affinities and influences, they do

42 _{Tutorial periods were missed due to the following external factors: two test weeks held during the semester (wherein} students do not attend lectures or tutorials), public holiday (Heritage Day), third-term holidays, localised flooding in October 2017 which caused damage to lecture venues, and one tutorial activity had to be repeated so that students attending from the other group could keep pace with the tutorial activity. As a result, at least six tutorial periods were lost during the semester.

97

not offer comment on the possibility that the affinities and system produced by the focus group in the first place may be flawed. Again, this underlines the critical importance of the execution and functioning of the focus group.

Although IQA calls for a single focus group session, in this study I had scheduled two sessions, five days apart, which allowed me to write up the affinity meanings (as described in 4.5.3) after the first session, for the group to check and confirm at the second. Their agreement would subsequently affect the SID, the interviews, and ultimately the findings of the study. In short, IQA protocol is not self- driving: the focus group phase needs careful planning and management, because all of the phases that follow hang on the credibility of the affinities. Practical considerations should not be overlooked, given the importance of this phase. I found the focus group sessions to be more time-consuming than I had anticipated, and at times logistically challenging, with 17 participants ranging back and forth along the array of cards attached to a wall. Allowing for two consecutive sessions might be an advisable adjustment to the IQA prescriptions, although it also increases the demands made on participants.

I believe the willingness of the participants in this study to commit so much of their time to the focus group sessions, subsequent individual interviews, and reflective writing, was due to several factors, including: the relationship I had established with the group over the semester; their relationships with one another, their understanding and support of the purpose of the research; an appreciation of having their views heard; and last, but certainly not least, the general improvement in understanding of content knowledge that they experienced in their working through of the tutorial activities. The level of commitment of the participants was essential to obtaining the depth of data which was generated in these phases of IQA.

The written reflections I requested of participants were an addition to standard IQA practice, which I hoped might offset some of the concerns around silence or conflict noted above, as a less focused and more private medium, reflective writing, might be a channel for insights students would not mention in the focus groups or interviews. My confidence in the affinities was increased when I found that the reflective writing could be accounted for within those categories of meaning. This modification to IQA may also have enhanced the quality of the data by predisposing the participants to think more widely or introspectively during the focus group phase, since they would have reflected regularly on their learning over the preceding weeks. A consideration of issues of representation in IQA highlights

98

similar concerns to those mentioned in the preceding paragraphs. Section 4.5.3 describes IQA’s reliance on composite quotes, woven together from individual sources to sound like a single voice, to elaborate the affinities. This practice presents a trade-off, where, while a holistic and nuanced picture is created for the group, individual voices are not reflected or tracked in these aggregated quotes.

The WhatsApp group chat that replaced the online discussion board as a medium of communication between myself and the study participants (see section 4.3.3), revealed conversation threads that were congruent with the elements that characterised the affinities. In particular, the elements of teamwork, cooperation and see-sawing emotions associated with disciplinary learning (to be elaborated upon in Chapters 6 and 8), were clearly evident when students helped each other with questions posted from past exam papers on the group chat. The range of emojis43 _{used to convey participants’ experiences’} of studying statistics or their gratitude towards their fellow colleagues for their assistance was enlightening. Thus the WhatsApp group chat served as a source of data triangulation through which I could confirm and further increase my confidence in this study’s findings.

IQA is about privileging student voices and minimising research influence. However, from my experience, the method as it stands already requires significant researcher engagement in deeper analysis and representation. While IQA is intended to be exactly replicable, this applies to the extent that different researchers presented with the same set of affinities and interrelationships would arrive at the same SID representing participants’ views of the phenomenon. Faced with an abundance of qualitative descriptions from the interview transcripts and reflective writing, I had to exercise my judgement, drawing on my own experience and understandings to select and compile composite quotes capturing the range of meanings participants ascribed to the affinities and their influences on each other. I am aware that in this sense my tracks as researcher are all over the data, and it seems disingenuous to suggest that the process of representation and interpretation in IQA is neutral, objective, or unequivocal beyond the production of the SID. Given this, I believe the occasional departure from protocol does not detract from the advantages of IQA as I have used it, but if transparently and reflexively undertaken, may serve to offset some drawbacks or shed light on some blind spots.

99

I reflected on my multiple roles as lecturer/tutor/researcher in Chapter 1. The different perspectives on students’ learning these roles afforded me assisted me in further analysing and interpreting participants’ descriptions of their learning. In a qualitative enquiry such as this, the researcher is a key research instrument (Creswell, 2013). I was aware of my own investment in the tutorial programme, and in the students’ learning. Teaching, as I experienced it in the tutorial programme, was an exciting and rewarding process. I came to know every participant and witnessed some of their learning struggles and successes at close range. Given that ‘objectivity’, in the sense of my being detached and disinterested in the participants and the course of their learning, was not possible, a supportive stance seemed appropriate.

Clearly I was not an objective outsider as just described. To some extent, the participants seemed to have constructed me as an insider, for instance, assigning mentions of me to the Tut Group, Journey of Understanding and Personal Journey affinities. For my part, beyond the distinct differences (in status, age, race) between the participants and myself, I empathised with them as students learning a new, sometimes troublesome discipline that might demand that they tolerate uncertainty, let go of old understandings, and reconstitute their identities, because this resonated with my concurrent experience in reading for a PhD. At the same time, obvious and undeniable power differentials remained in my role as their lecturer (and eventual examiner). Using IQA as a research approach helped to reconcile these positions, and mitigated my own discomfort around the split roles I occupied. By casting the participants as the experts on their own learning, IQA remained consistent with the dismantled authority relationships which characterised the cooperative learning pedagogy of the tutorials.

4.9 Concluding comments

This chapter has described the research design and methodology I used in seeking to enhance understanding of the processes and experiences of students’ learning in statistics: a case study within a qualitative, interpretive paradigm, involving participants in a threshold concepts- enriched tutorial programme. The chosen approach of Interactive Qualitative Analysis was explained. I also considered issues of rigour, and the ethical considerations affecting this study, before offering some reflections on the methodology, and on my role in the research. Chapter 5 provides detailed description of the application of IQA in the focus group sessions, and presents the SID, which represents the group’s view of their learning in the threshold concepts-enriched tutorial programme.

100