Figure 4.7 illustrates the timeline for the implementation of the different research instruments in each phase of the two pilot studies and the main study.
166 Two pilot studies (see Section 4.7) were launched during the first and second semesters of 2015. These pilot studies were followed by the main study in semester 1 of 2016.
Figure 4.7: Instrument-timeline of two pilot studies and main study.
(Qualitative)
167 4.7 THE PILOT STUDIES
4.7.1 Definition and aim
Maxwell (2008) regards a pilot study in qualitative research as a useful opportunity to interpret and understand the perceptions and actions of the participants being studied.
Indeed,
“in a qualitative study, these meanings and perspectives [of participants] should constitute an important focus of your theory … they are one of the things your theory is about, not simply a source of theoretical insights and building blocks for the latter” (Maxwell, 2008:227).
In other words, piloting gives the researcher the opportunity to also consider and reflect upon other conceptual resources apart from literature. A preparatory literature review revealed that most mathematical modelling contexts specifically referred to implementation in developed countries or in high school education; therefore, lacking clear indicators of how a modelling approach would play out locally, how it would be received in a local setting and whether the research protocols would be practical. In Social Sciences research, a pilot study often focuses on a single research instrument (Van Teijlingen & Hundley, 2001). To test the feasibility of a study, a pilot study may commence with a “qualitative data collection and analysis on a relatively unexplored topic, using the results to design a subsequent quantitative phase of the study”
(Tashakkori & Teddlie, 1998:47). Since the mathematical modelling workshops constituted a novel intervention at UJ and further afield in SA, it was decided to launch a separate trial to test only the qualitatively-driven mathematical modelling workshops (pilot study 1) before a full-scale quasi-experiment was launched (pilot study 2).
4.7.2 Pilot study 1
Pilot study 1 was not aiming to answer any specific research question. Rather, through this study, the feasibility of various design aspects could be explored, students’
perceptions of the modelling instruments could be gauged and the adequacy of the data collection method could be confirmed. The aim of the preparation work was to effectively assemble separate ideas borrowed from literature, to construct a workable blueprint, to dispel personal concerns and to critically examine the process prior to the rolling out of the main study (Maxwell, 2008).
168 It seemed attractive to involve an entire cohort in pilot study 1. Had only a small-scale trial be launched, the resulting data could have been too limited to make conclusive decisions on future implementation into the main study. Pilot studies are particularly desirable when testing an intervention. The notion of piloting originated in the medical field where the testing of interventions are costly; therefore, small-scale testing is generally regarded to be a sensible prelude to the roll-out of a larger study (Polit &
Beck, 2004). These authors critique the misinterpretation of the term pilot study:
a pilot study is not the same as a small-scale study … the purpose of a pilot study is not so much to test hypotheses, but rather to test protocols, data collection instruments, sample recruitment strategies and other aspects of a study in preparation (p.196).
Pilot study 1 involved two mathematical modelling tasks and an open-ended questionnaire, all completed by the entire cohort. The decision to use a cohort was also inspired by accounts where small-scale undertakings fall short in affording remedies to problems that transpired later-on during the full-scale study (Makonye, 2011). Largely, no major problems were encountered during the piloting of various instruments. Data collected from the pilot studies were helpful and also contributed favourably to the inquiry (Arain, Campbell, Cooper & Lancaster, 2010).
Participation in pilot study 1 was compulsory for all Engineering Mathematics 3 students (n = 93) registered during semester one of 2015. Two mathematical modelling tasks were scheduled to take place towards the end of the semester when students were better acquainted with Mathematica technology and the theory of DE (Galbraith
& Stillman, 2006). The mathematical modelling tasks were students’ first exposure to this problem-solving approach. Traditionally, group work is rare in engineering mathematics modules at UJ– even in the computer laboratory – mostly due to time constraints. Recent Mathematica test scores were analysed to create heterogeneous groups of five to six students per group. A 150 minute laboratory session was allowed for each task. Tasks were completed during two once-a-week laboratory sessions, on 28 April and 5 May respectively. Of the 93 registered students in semester 1, 80 (86%) attended the two workshops. During pilot study 1, students also completed an open-ended questionnaire.
169 4.7.3 Pilot study 2
Pilot study 2 was launched during the second semester of 2015 and a new cohort of 130 students formed the sample. The design for pilot study 2 involved three sequential quasi-experimental phases. Phase 1 included a pre-test and was followed by a demographic survey. For phase 2, volunteers were invited to attend two mathematical modelling workshops. Semi-structured reflective group interviews were conducted after each workshop. Finally, data was collected during Phase 3 with a post-test. The pre-test and post-test were administered to the entire cohort. The pre-test was written at the same time as the second Mathematica semester test, while the post-test was written along with the Mathematica examination paper. This way, a natural assessment environment was created with minimum inconvenience to students. The experimentation did not affect the natural rhythm of normal curriculum activities and students appeared to be unperturbed by the (unusual) research activities which could play out in natural settings. However, participation in the modelling workshops was voluntary. Students were briefed and had been given a week to decide who wanted to volunteer for participation in the intervention. Fourteen of the 130 students volunteered to participate in the mathematical modelling intervention and were divided into two groups. Table 4.3 shows data of the fourteen students in pilot study 2.
Table 4.3: Grouping of participants in pilot study 2.
Mathematica Test (%) Group 1 (n = 6) Group 2 (n = 8)
Columns two and three of Table 4.3 show the amount of students in each group who attained a specific Mathematica test score as indicated in column one. The two groups remained intact for both tasks. The first workshop took place on 6 October during a scheduled practical period and the second workshop was hosted ten days later on 16
170 October (after hours) when it was convenient for all participants to attend. The modelling tasks remained the same during the two pilot studies. The fourteen participants were invited to semi-structured reflective group interviews after each workshop.
4.7.4 The modelling tasks: design, aim and appropriateness
The two modelling tasks were designed with the aim to explore engineering students’
visualisation and experiences when engaged in a mathematical modelling approach.
Setting the modelling milieu according to RME principles, the tasks aimed to elicit:
Students’ metacognition through “connection questions” (Kramarski & Hirsch, 2003:43)
Students’ actions, processes, formation of objects and schemas on paper and pen and with CAS as proposed by APOS Theory with attention to the
o reversal of an existing process (Duval, 2006)
o formation of new processes by coordinating two or more processes (Dubinsky & McDonald, 2001)
Students’ intuition by relating to their own real world experiences (McDowell, 2006;
Prusak et al., 2012)
Students’ current mathematical knowledge and technological skills (Galbraith &
Stillman, 2006).
4.7.4.1 Task 1
During the first workshop, each group had to solve a modelling task; this will be referred to as Task 1. The task involved a 2011 National Geographic article10 that was selected to elucidate the real world applicability of DEs as models of population growth. A hardcopy of the magazine article (see Appendix B) was handed out to all participants.
The article represented the real world problem in the modelling cycle (see Figure 1.1).
Students were given ample time to read the article by themselves. They were encouraged to underline or highlight important details and make annotations in order to understand the context of the article. After students had sufficient time to study the
10 Link to National Geographic article: http://ngm.nationalgeographic.com/2011/01/seven-billion/kunzig-text
171 magazine article, the modelling task, in the form of a worksheet (see Appendix C), was handed out. The worksheet outlined the world population data11 between the years 1900 and 2000. A table of 20th century data (see Appendix C) was provided to students as part of the task. For this purpose, real world data were arranged over the period 1900 to the year 2000. The problem statement for this task was: Model a differential equation that represents the 20th century data in the table.
The worksheet contained four sub-sections. Accordingly, the task was delineated into sub-tasks by using “connection questions” (Kramarski & Hirsch, 2003:43) as follows:
Planning the solution – in this section, groups were advised to make notes, sketches or diagrams to structure their thoughts about the given information (Kramarski & Hirsch, 2003; Mevarech et al., 2006). To this extent, groups had to:
o ask themselves what the problem was about and how you could best approach it
o think of any mathematics they had ever done before that could be used o ask themselves how this problem was perhaps similar to other problems
they had solved previously
o write down what they intended to do step-by-step
o discuss their plan as a team till they reached consensus
Execute the plan – in this section, groups had to follow the steps they set out to do in the planning phase and ask themselves what kind of strategies they could use to implement this plan
Reflection – in this section, groups had to ask themselves whether their solution made sense and whether they had answered the problem completely. They had to critically look for anything that didn’t fit or seemed to be incorrect. If so, they were advised to go back to the planning phase and check where they needed to make adjustments
Validation – in this section, groups had to check the validity of the real world solution and explain what the model meant. As indicated in the given data (see Appendix B), the population in the year 1900 was inaccurate. Groups had to recommend and substantiate a more appropriate initial condition for the year 1900.
11 World population data, available from http://en.wikipedia.org/wiki/World_population_estimates
172 It was also required from groups to extrapolate the data and make inferences about the missing data point in the table that corresponds with the population in the year 2020.
Task 1 was delineated to help avoid bottlenecks (Galbraith & Stillman, 2006). Table 4.4 reveals the respective subtasks with a rationale for each subtask and an expected response.
Table 4.4: Task 1, delineated into subtasks with rationale and expected response
Subtask Rationale Expected response
1. Use Mathematica to plot the
Reason visually Making assumptions;
curve recognition 4. Write down a formula (equation)
that can represent this function
Connect graphical model with symbolic model
Translate from visual to symbolic representation 5. Use this formula to derive a first
order differential equation (DE) that can model the real world data
Recognise/apprehend the origin of a DE
Construct a DE based on real world data
6. Use the given real world data (wisely!) to find the parameters of your DE
8. Make a recommendation on the population count in the year 1900 according to your model
Critically reflect on validity of initial condition of DE steps you followed, highlight specific difficulties you may have encountered
173 At the start of the first workshop, students were informed that there were no rules, no
‘correct’ answer to the task, that any source of information whatsoever may be used but that each group must safeguard their work and not share it with any individual outside their group. To create an informal and relaxed milieu, chairs were rearranged to form small workstations. Small groups were dispersed around the laboratory, leaving space for private discourse and freedom of movement within designated isles.
A pseudo-laissez-faire12 style was adopted – the lecturer became a facilitator and spectator. Blum and Borromeo Ferri (2009:52) advocate a “learner-oriented classroom management” whereby the lecturer stimulates students’ cognitive and metacognitive engagement in activities. The role of the lecturer was restricted to subtle interventions.
A nuanced balance between teacher guidance and student independence was key.
The nature of interventions by lecturers during the modelling process is non-trivial (Kaiser & Stender, 2013) since interventions mostly require impromptu and intuitive judgments. Ang (2013) warns about constant intervention and over-facilitation. The role of the lecturer was consistent throughout the workshops, both in the pilot studies and the main study.
On completion of the task, each group had to submit their documents; these included hardcopy documents as well as electronic Mathematica or Excel files. All groups in pilot study 2 could finish Task 1 within the available 150 minutes, but of the 15 groups in pilot study 1, six groups only submitted the next day. Students employed an unanticipated variety of sources; these included You Tube videos, laptop computers, paper-and-pen rough work, pocket calculators and textbooks.
The task design was an important aspect of the empirical leg of this study and was complemented with a suitable CAS learning environment in the computer laboratory.
The gap that was identified in the current body of knowledge referred to the absence in research and practice of mathematical modelling and visualisation in a CAS learning environment within the vocational stream. The mathematical modelling learning approach of Task 1 therefore aimed to connect the mathematical world, the technology world and the real world and is illustrated in Figure 4.8.
12 Laissez-faire: From the French, a style where people can do as they please.