No. Question Theoretical framework Data analysis Representation of outcomes
1 How do students
exercise shared epistemic agency?
Epistemic agency is expressed in the intentional object-oriented measures that groups or individuals take to build or create knowledge (Paavola & Hakkarainen, 2005; Scardamalia & Bereiter, 2006).
Synthesis of findings of questions 1.1-3. A narrative of each group's project from the perspective of epistemic agency, built through representations of the subquestions in this section and tied together by discussion sections that compare the ways epistemic agency manifested in the cases, grouped by task. Descriptions of points or episodes in which infrastructure tended to affect students' epistemic agency - from student reflections, records of discussions and observation.
1.1 What influences how infrastructure is assembled and used?
The dispositions (Perkins et al., 1993) of students in framing the task and collaborative work, and hence how they set up their project infrastructure. The infrastructure that students build to support collaborative work undergoes modification through practice and is designed in use (Goodwin, 2013).
Frames, focus, evidence of dispositions in each group.
Episodes where there is mention of, or action towards, how to collaborate and what tools to use.
Student interview answers on their initial views of the task.
Description of how each group has initiated and set up their activity infrastructure.
1.2 What secondary infrastructure do students assemble to support knowledge-based tasks?
Following Nicolini et al.'s (2011) framework, this question focuses on the secondary level infrastructure of task- relevant and situated tools and resources, with some reference to tertiary
(background, ubiquitous) infrastructure.
Descriptive coding of student activity, based on the elements identified as integral to epistemic agency: actions, objects, visible infrastructure. How and when groups use shared tools. The physical setting and students' orientation towards each other and tools or objects. Where things break down, or are abandoned.
Representations include: comparison tables of practices/tools/objects and mode of use across groups; excerpts from observed student interactions; quotes from interviews; screen shots, diagrams and photographs of shared work spaces.
1.3 In what ways do students employ epistemic objects for knowledge creation?
This question addresses Nicolini et al.'s (2011) primary level of infrastructure, the use of objects in knowledge creation. Students develop epistemic objects and use technical objects when creating knowledge. Objects, through their properties and affordances are 'actors' in knowledge creation (Ewenstein & Whyte, 2009; Nicolini et al., 2011; Pickering, 1993).
Focus on interactions - how the elements come together in knowledge creation. Actions and discussion focused on objects. How ideas and changes are acknowledged or discarded; what type of discussion is generated.
Modes and features of tools and objects used in knowledge work.
Declarative and procedural knowledge used; recognisable epistemic games.
Comparison and contrast between cases.
A 'life story' of each group's overall epistemic object, including the role it and its components each has in developing knowledge. This includes diagrams, textual description and images.
The changes to objects and their roles over time. In-depth descriptions of key episodes of object-oriented interaction.
A section defining and justifying the concept of synthesising objects.
2 What does
students’ activity suggest for design for epistemic agency?
Observing student practice is necessary to inform learning design (Buckley et al., 2010; Greenhow et al., 2009).
Students benefit from guidance on both the practical methods of collaboration (functional actions) as well as how to work together to create knowledge (epistemic actions) (Damşa et al., 2010; Muukkonen et al., 2010).
Students' preferences can be incorporated into learning design and their lack of experience augmented by supplying cues and modelling effective practice.
Summary and synthesis of findings from questions 2.1-2.
Based on the outcomes of the sub-questions below, refine principles for learning designers in supporting student epistemic agency.
2.1 What is present or missing from the group projects that learning design could address?
Concepts of epistemic agency and knowledge creation, in the context of learning design principles in Chapter 6. As for 2.
Observable conditions and actions in student groups that aided or hindered their response to tasks.
Explicit use or mention of task instructions, supporting resources, suggested tools and ways of working.
Comparison of groups' ways of collaborating and answering the task.
Variations on use of technologies and practices between groups.
Analysis of each case for areas that were or could be influenced by learning design.
(Collections of referenced discipline and general knowledge touched upon by the engineering groups - appendix).
2.2 What implications does this have for design of knowledge- creation tasks and environments?
As for 2. Compare the results of the above exploration with extant approaches to learning design, and synthesise a general set of principles for supporting epistemic agency.
Practical design recommendations for supporting epistemic agency in student group collaboration, illustrated by case examples.
An ethnographic approach, especially one using video and audio recording, entails a large quantity of detailed data. The set of data is a valuable record of people going about their work, but much of it has to be, if not discarded, summarised or selectively shared.
Every ethnographer is painfully aware of the discrepancy between the richness of the lived field experience and the paucity of the language used to characterize it. There is necessarily a dramatic reduction, condensation, and fragmentation of data.
(Bruner, 2001, p. 144).
Measures I employed to manage volume and keep analysis linked to data included connecting pattern and narrative to explicit examples, summative mapping of projects and comparative analysis between cases.
I employed analytic methods under the general approach of discourse analysis, taking multiple passes over each case, using:
• Multi-modal transcription and notes • Descriptive coding
• Diagrams to summarise case projects and shared epistemic objects • Written narratives
• Comparative analysis of findings, connecting statements and principles with case examples Details of methods are outlined below.
How can this study be evaluated, considering that “[t]he evaluation of qualitative research is a complex and disputed area” (Taylor, 2001b, p. 41)? Greenhalgh and Swinglehurst (2011) name three key interpretive criteria in judging ethnography: “authenticity,” “plausibility,” and “criticality” (p. 4). The evaluation, of course, is made by the reader, who brings their experience and expertise to this judgement.
On the first criterion of authenticity, described as “immersion in the case through extended fieldwork” (Greenhalgh & Swinglehurst, 2011, p. 4), I transcribed6, coded and analysed recordings
and online communications and documents, resulting in a close familiarity with each case and its development over time. I reviewed each group’s case a minimum of four times, using different means of analysis, with intervals between each revisit. Case details, vignettes and explanations are situated in “the specific circumstances of place, time and participants” and contingent on those (Taylor, 2001a, p. 319). The diversity of data, inconsistencies of natural talk (Taylor, 2001a) and details of cases contribute to the authenticity of the study.
On the second criterion of plausibility, “developing explanations of local phenomena which made sense to participants and drawing these together into a coherent narrative” (Greenhalgh & Swinglehurst, 2011, p. 4), I focussed on systematically making meaning from the assembled data. The sense of how participants understood the situation was gained primarily through analysis of their discourse in the moment; student interviews provided general and retrospective confirmation and further individual perspective. Following on from descriptive coding and diagrams, a key process of analysis was constructing the narrative of each case; moving back and forth between
transcriptions, images, objects and diagrams to test and strengthen representation of data. To add to the sense of participant sense-making, plausibility is also established by the reader of the study connecting it with their own experiences of similar situations: “readers test [research reports] in application to new data in the very process of reading” (Katz, 2001, p. 213).
The third criterion of criticality, “systematically questioning taken-for-granted assumptions” (Greenhalgh & Swinglehurst, 2011, p. 4), was exercised with each case description and through comparative analysis of cases in the context of identified theory and design principles. My research questions and theoretical framework, which expanded in response to identified phenomena in the data, have guided what elements to describe and which episodes to treat in detail. Selection often focused on where the object of activity was ‘problematized’ (Engeström & Toiviainen, 2011), that is, when students identified and discussed issues or concepts and planned activities. I have
endeavoured to follow Silverman’s (2014) advice and “assemble fragments of everyday
understanding” (p. 429). My case descriptions kept to the reality of what I observed rather than fitting them to preconceived notions, although they were theory-informed. I used analytic research, in which I sought out contradictory data to refine analysis, thereby leading to “a holistic analysis that binds propositions and data into an intricate network” (Katz, 2001, p. 208). I reviewed data to find repeat examples of the phenomena I described, looking for negative examples and the contexts in which those phenomena were not observed. Making sense of the data, writing and representing it, was part of analysis: “the act of writing itself is a way of thinking—and of knowing—about our work” (Gullion, 2016, p. 107). Ethnographic writing is expected to evoke the people, places and activities it describes, although it is not simply storytelling: “we also theorize through our narrative forms” (Gullion, 2016, p. 8).
I have built upon detailed immersion in the cases a “density” of data and analysis:
“For each qualitative field report, readers can assess how richly the researcher has perceived internal variation in the data; how radically the researcher varied his approaches to subjects; the density into which data and analysis have been interwoven; and the practical ease of testing the theoretical claims on new data” (Katz, 2001, p. 216).
It is not an issue of the exact reproducibility of the results, but that the methods can be used by others to extend exploration. If another researcher were to study similar groups using the methods used for this study, they would be likely to identify similar phenomena. Evaluation also includes the element of application to practice: is it ‘fruitful’ and can it be used by practitioners (Taylor, 2001a)? I used the frame of learning design for findings in this study and how those findings can be applied to higher education practice. Evaluation comes down to the question, ‘Will an expert reader of this study have confidence that it is authentic, plausible, critical and applicable to practice?’
3.3.1 Discourse analysis
The discourse in this study is that of specific tertiary student project groups, evidenced in their interactions in person and online, as well as the objects they use and produce. In the sense of a point of view that is embedded and perpetuated or “taken for granted” (Atkins & Wallace, 2012, pp. 169–170), the group discourse is also situated within the discourses of tertiary education, including student/lecturer roles and understandings of formal education.
“[W]e will find that the discourse conventions of learning and teaching are deeply embedded in our cultural consciousness. We have particular expectations of what teachers and students should do (and not do) as well as what they may say (and not say)” (Woods, 2006, p. 156).
Discourse analysis is used here to identify the meaning in the interactions between students as well as their overall framing of the situation and their activity.
I use discourse analysis to investigate the “language plus context,” in which context “includes our experience, assumptions and expectations” (Woods, 2006, p. x), but also encompasses objects and activity on and around them. Discourse shapes and is shaped by its context: “the world” or
surrounding environment, language used, participants and their relationships, “prior discourses” that shape our expectations, the medium of communication, and purpose (Johnstone, 2008, pp. 10– 19) —these elements provide a set of heuristics for conducting discourse analysis. For example, I have noted how: students spoke from the perspective of members of an engineering company; suggestions were taken up or ignored; project approaches seemed to conform to accepted patterns of assessment; and how the format of tools and artefacts influenced collaboration and knowledge creation. This is a two-way influence: while discourse is shaped by existing conditions, it produces its own meaning in the understanding participants build between themselves. I use a combination of methods, described below: multimodal transcription; descriptive coding; diagramming of objects and activity; and case descriptions that focus on the sociomaterial activity of the groups. Table 3 outlines how data analysis for each question focuses on particular elements generated by these methods.
3.3.2 Multi-modal transcription
While the selection of methodology and data capture methods set out the field of research, transcription was the main starting point of analysis (Taylor, 2001b). Transcriptions use common markup conventions developed by Gail Jefferson and commonly used in conversation analysis (See Mondada, 2011). See Appendix 5 for transcription protocols. I recorded a close to verbatim
transcription of each utterance and quoted speech is as spoken, without correction for grammar or usage. The volume of recordings and methodological lack of need for word-by-word accuracy meant that a fully verbatim transcript was not practical or required, understanding that “[t]he level of detail you need in your in transcripts will depend upon your research problem and your preferred
analytical approach. Practical issues, such as time and resources are also relevant” (Silverman, 2014, p. 333). The transcriptions recorded necessary detail to answer my research questions.
In transcription I paid attention to the sociocultural setting and “contextualization cues” (Gee & Green, 1998, p. 122; Gumperz, 1982) by noting speaking tone, actions, gestures, tools and resources in addition to dialogue. While transcribing interactions, I took brief notes on what stood out for me during that first listening, in the context of how the students were organising their work, what they were focusing on, what tools and objects they used, and how they developed shared objects. I treated online communications as part of discourse and incorporated them in the chronology of interactions.
3.3.3 Descriptive coding
After transcribing student interactions, I re-read the transcripts to add further descriptive coding and identify similar and contrasting episodes within and between cases. Descriptive coding is
recommended for ethnographic studies7 (Saldaña, 2016, p. 102), and I used it to organise my
understanding of what was occurring. The codes were “broad and overlapping” rather than “exclusive coding categories” (Taylor, 2001b, p. 39). I freely used appropriate single words and phrases rather than restrict myself to predetermined codes or wording. I took Silverman’s (2014) advice to use what the participants and data are saying, the categories that are suggested through dialogue and activities, rather than retro-fit an externally-created set of codes. However, Damşa et al. (2010) contributed thematically by identifying categories and examples of epistemic actions: “creating awareness,” “alleviating lack of knowledge,” “creating shared understanding” and
“generative collaborative actions” (p. 175), guiding conceptualisation of phases and types of activity. They also identified process-related actions in collaborative work, related to project goal-setting, regulating shared work on objects and personal interactions. I arranged descriptive coding within the initial general areas (frame, object, knowledge, tools/infrastructure, process, group
management) that I wanted to explore, as well as concepts that I found were repeated within and between cases.
I coded in Excel workbooks, with one workbook per case, one spreadsheet per collaborative session, one conversational turn and speaker per row, and seven themed columns for descriptive coding (see Figure 2 and Appendix 6 for samples of transcripts and coding). The columns corresponded to elements required to answer the research questions. The first column was ‘frame,’ noting the points where a new frame occurred and related to sometimes long passages of discussion. Framing is used as a way of understanding how students were making sense of the task through their immediate focus and is evidenced in what the students do and say. Episodes of framing occurred between the points at which focus or topic changed.
After ‘frame,’ the next column was for ‘object/concept’, which acted as a type of sub-coding for framings in which I aimed to capture the focus of each short episode, delineated by a movement between different concepts or activities. Another column was used for ‘actors’ other than the students, including tutors but mostly non-human actors such as the modes of communication, tools, resources and information sources that the students activated, that is, when infrastructure became visible. I included the tutors as part of the infrastructure where they were deliberately called upon by the students. The column initially dealing with ‘knowledge work’ changed to ‘epistemic games’, focusing less on the step-by-step actions and more on the type or pattern of work in which the students were engaging. The next column captured actions related to managing the group and project. A further column held free notes and the last identified social conversation and personal experiences. These columns facilitated a rough mapping of the ‘landscape’ of each group
collaboration and subsequent comparison between groups.
7 Saldaña does not recommend descriptive coding for case studies, but my study uses ethnographic