Early approaches to data collection

In this study, the research participants were undergraduate engineering students at Victoria University in Melbourne, Australia, taking a syllabus that made regular use of PBL. The goal was to adopt a neutral participant observation where the researcher’s role is solely to observe and record the different ways in which the PBL students interact with each other using the various meaning-making resources available to them or created by them in the context of a PBL task. Although the students had given their consent to be video-recorded, the researcher needs to be

prudent in keeping the identity of the participants anonymous and to give the participants non-identifying names when conducting multimodal transcriptions or use numbers to indicate them and not share their identity with others. In addition, student participants were reminded that they could withdraw at any time, leave at any time and that any data involving the individual who did not wish to participate further will be discarded from the study. Student participants were also informed and reminded that the researcher was not a teacher, and would not direct, instruct or help them with their work.

However, the research reported in Chapters Four and Five was not the first set of observations by the researcher. Two earlier PBL sessions were video-recorded and then discarded as not providing a suitable basis for this type of study.

The initial phase of data collection took place in 2009 in the first semester of the first year of an undergraduate PBL class. Initially it was hoped that this round of data collection would be used in the main study, but in reality this preliminary stage of data collection became an opportunity to test the planned approach to data capture and transcription and to lay a basis for future recording and analysis. The aims of the pilot were:

1) To apply and assess the planned approach to video-recording and observing PBL learning in action;

2) To transcribe the data into a format suitable for multimodal analysis, including the use of image capture and video editing software; and

3) To undertake a preliminary analysis of the data to determine its capacity to enable the planned research questions to be answered.

Following ethical approval, it was agreed that recording would focus on two teams of PBL students enrolled in a unit titled “Problem Solving for Engineers”. This was the students’ first experience of PBL and each task was short-term and relatively well defined with the aim of introducing the learning methodology and assisting the students to become familiar with it. The first PBL team consisted of five students, four male and one female. The second PBL team consisted of four students, all being male.

Figure 3-1: First student group

Figure 3-2: Second student group

The PBL teams were from two different classes, but with the same facilitator tutoring them. The choice to capture different groups with the same facilitator was made because it would enable a potentially valuable comparison of differences in group approach to problem-solving, whilst also having continuity in the facilitator’s pedagogical and communicative strategies.

The PBL problem the students were set was to work out how to determine the height of a sports oval light tower using a theodolite and trigonometry and to introduce them to the concepts of engineering surveying. Furthermore, engineering

students will learn through conducting this problem-based activity some geometric and trigonometric laws and rules which facilitate the calculations to be conducted. The facilitator’s role was to instruct the students in how to use the theodolite in a technically competent way, and then apply the measurements that they derived to calculate the height of a sports field oval light tower. This is why the students conducted their PBL task outside of the classroom in a park adjacent to the campus. The engineering PBL student teams used different resources to conduct their task. Some of these resources were defined by the task such as surveying tools including digital theodolite, tripod, box tape, hammer, pegs and nails. Some resources were used by the students to record their measurements and therefore to achieve their task such as notebooks, calculators and pens.

The purpose of this stage of data collection was to develop a method for multimodal data transcription. Of particular value was the need to consider how to code and interpret the use of resources and how the PBL team interacted with the different surveying instruments (such as the theodolite) to make meaning and hence solve the problem, which is finding the correct answer of a sports oval light tower’s height. However, the data obtained was not sufficient to be the main source of the research data. This is because the problem was relatively easy and, in consequence, there was very limited development of meaning making by the students as the nature of the task meant this was mainly the application of technical skills.

The second phase of data collection also took place in 2009 in the second semester of the first year of an undergraduate PBL class. This iteration in the research design was specifically designed to address some problems in the initial phase. In particular the focus was on a PBL class dealing with a deeper and more complex problem. Specifically, the goals were:

1) To obtain sufficient data to be used as main data source for the research; and

2) To avoid the technical difficulties faced by the researcher in the initial phase, such as voice quality, how to follow the students and to focus the camera.

One team consisted of five students (two were female), selected with the help of the researcher’s then associate supervisor who arranged the students’ consent to being video-recorded for the research purpose. This group was expected to yield richer data

1) The students were more active and more engaged in the previous PBL problem/task compared to other PBL team in the class; and

2) They also were more independent in learning than other PBL teams. In this case the recording was easier as the PBL team met in the classroom and laboratory to work on the problem. For this module, the students were presented with the more substantive task of designing a building to grow crops, taking account of heating, water supply and temperature regulation.

Figure 3-3: Second phase: Student group in laboratory

However, although this phase was designed to compensate for the initial difficulties it was not completed because some new problems and difficulties arose:

1) The facilitator was not helpful in allowing access to the necessary teaching materials about the requirements of the PBL task. This removed valuable contextual knowledge that lessened understanding of the PBL team’s interaction and interpretation of the main semiotic resources utilised by the students;

2) In the laboratory, the PBL team tended not to work as a close group and instead the students spread themselves out. Therefore, it was difficult to record them all at the same time, using one camera. The students themselves indicated they were uncomfortable with being recorded despite having given their consent. As a consequence recording was sequential, with the video recording showing one sub-group and then another which meant valuable information was missing and there was less evidence of all-group problem solving; and

3) In addition students from other PBL teams interacted with the main group and since they had not given their consent to being recorded, it was not possible to use these excerpts.

Because of these difficulties, the stage of data collection was cancelled after one month of observation. However, this stage was valuable in refining the data collection approach and these lessons were applied to the final stage (which is the material reported in Chapters Three and Four).