Data analysis

The purpose of the data collected in each case is to help produce both an overview and a highly detailed description of the developing skills and knowledge of each student, through the process of analysis and interpretation. The data was analysed in multiple stages, starting with data shared by all cases (school-specific data, observation data) before progressing to individual data (interviews and artefacts) and then finally to a comparison across the cases. A narrative of each method of analysis is given here, and summarised in the tables in Appendix C.1.

The lesson plans collected as lesson-focussed data were compiled and analysed in two ways. First, the project overviews were annotated and coded according to objectives. The primary interest was whether the tasks expected of the students were leading to a change in foundational or conceptual knowledge, procedural skills and knowledge, or transferrable skills and knowledge. Student motivation was also considered. These were coded independently by me, although in several places, the instructor added their opinions on the nature of the day’s instructional aims. These codes were derived as a compromise between Anderson et al. (2001)’s revision of Bloom’s taxonomy, and Fink’s (2013) taxonomy. A table of codes can be found in Appendix C.2. It should be noted that the coding for the lessons was more general than the coding for the interviews; the code groups rather than the individual codes were used.

Before codes could be determined, it was necessary to develop a strong familiarity with the data. As mentioned in discussion of the data collection methods above, the transcription itself served as an initial round of analysis, from which codes could be derived based on themes derived from the text (emergent coding). Under the general headers relating to content

knowledge, procedural knowledge, transferrable skills, and motivation, codes relating to

examples of learning under these categories were determined. A fifth category, conceptions, was used for the Water Management Project, but not for the Computer Science one. This is because the aims for the CS project (as laid out in the project brief) prioritised the other forms of

knowledge, and familiarity with both CS students’ cases suggested their learning had been focussed on procedural knowledge and transferrable skills, rather than content-based concepts. This is not to say that no conceptual learning took place, but rather to suggest that learning in other knowledge dimensions (Anderson et al., 2001) was the focus of this project.

The same coding scheme that was used for the project briefs was also used to code all lessons and artefacts produced by the students when looking at the learner-focussed data. This was done because the second research question deals with the link between the lesson elements and the student learning, but also because it may reasonably be expected that teaching objectives are written with the intent of leading to learning of a similar nature. What I discovered quickly was that there were many instances in which the same code could be applied; this was not unexpected. Because the purpose of the codes was to act primarily in the manner of an index (Bernard, Wutich, & Ryan, 2017), and because no two examples of, for example,

of the coding was to group together portions of the data that shared similar properties for the second phase of analysis, and they were used successfully in this manner.

The second phase of learner-focussed analysis involved grouping selected quotes by code in separate Word documents, in order to allow for the analysis of specific content categories. The determination of which code groups to select was the most subjective decision of the entire analysis process; while it was based primarily on relative frequency of code use, it was also based on the nature of each student’s narrative (Lieblich, Tuval-Mashiach, & Zilber, 1998). Jane for example had stated that her role was that of report writer and organiser, rather than

programmer. The most meaningful learning for her was more likely in the areas of consumer focus, communication, and planning than in programming logic and knowledge of syntax, though these areas were not entirely abandoned. These codes allowed for the formation of categories within the data, which could then be separated out and considered independently before being considered in conjunction with one another. In the cases of Hannah and Jane, their data could also be used as a means of member checking, as they were part of the same project group and could offer perspectives on the same events, and so their narratives relating to particular incidents were often considered together.

The result of this second phase of analysis was the decision to document Hamish’s conceptual changes as his primary form of learning, while for Jane the primary focus was on transferrable skills. Hannah’s learning was harder to classify, for the conceptual understanding she developed during the project manifested primarily in procedural knowledge due to the nature of her tasks. This means that, while the focus of her narrative is presented primarily as

procedural knowledge, it is inextricably linked with her increasing conceptual understanding as well. For Hamish and Jane as well, learning took place across all of these dimensions, but in order to allow my narratives to remain faithful to theirs, a decision was made to allow the nature of their stories to determine what formed the focus of their learning in their projects. These foci were determined through the establishment of multiple documents for each learner, based on the categories established above. The analysis of these involved comparison to the baseline gathered in the initial interview, and from that baseline I looked for changes in complexity of conceptual knowledge/applied task, or integration into other areas. It was evident from this second round of analysis that the focus and nature of learning for each participant was quite different, and so one or two of the potential learning categories (examples include consumer focus, programming

logic, conception of biodiversity) for each participant was selected for further analysis. This involved in some instances a subjective determination of learning, based on my familiarity with the participants. This is in line with the interpretivist framework adopted for the methodology.

The above phases resulted in sufficient answers to the first research question, relating to how students learn over the course of a PBL project. In the final phase of the data analysis, the lesson-focussed and learner-focussed data were considered together to evaluate what impacts the lessons themselves had on learning, as a means to answer research question two for each case. This is not to suggest that either was ever viewed in true isolation; the preliminary analysis (transcription phase) informed the coding and analysis of the lesson-focussed data, which in turn then influenced to a degree the coding and analysis of the interviews and artefacts. This was not only the most efficient method I could contrive for the analysis; it was also in line with the interpretivist framework I had adopted for this study.