Chapter 3. Methodology
3.6 Sample
For the pilot studies, the participants that were used were from a purposive
sample and intended to test the instructional materials, the tasks, the software and the
remote observation method. The participants chosen for the pilot studies were all
postgraduate students doing either their masters or PhDs. For Pilot Study 2, the 6
students who participated were all from the Open University. These students were
approached personally by the researcher and asked whether they will participate in Pilot
Study 2. After these students showed an initial interest, an email invitation was sent to
them. For Pilot Study 3, three students were also approached personally and were asked
to participate in the study. This time only one student was from the Open University and
selection of participants in the Main Study which is further explained in the next
section.
In the Main Study, based on the experimental design, the expected number of
students was 36, although data from 38 students were collected (see Section 5.2.1,
p.125). Whilst it might be argued that this yielded only a small number of participants,
the sample size of 38 students was comparable to those used in other experiments on
mathematical problem-solving at campus-based universities. For example, Renkl and
Atkinson (2003) reported sample sizes of 34, 54, 45 and 28 in experiments that they
conducted over several years.
Further, using a software developed by Lenth (2006), it was determined that 12
students for each software box (that is, 36 students in all) should provide a statistical
power of more than 0.8. Power indicates the ability of a statistical test to find a
difference, if there is actually a difference and it is dependent on the number of groups
and the sample size (Howell, 2002). Power values of over 0.8 are generally considered
to be good.
3.6.1 Accessing Students via the Internet
Initially, these students were expected to be UK undergraduate students as the
research main concern was investigating university students’ learning mathematics.
Thus, a wide sample of UK undergraduate students was intended to be recruited as this
would ensure that the results would be more generalisable. The UK students were
recruited through social networking websites and students forums such as Facebook and
the Student Room.
It was expected that recruiting students from their popular social networking
sites such as Facebook would be easy. However, this was not a straightforward or viable
to post a recruitment notice on their forums, only 9 gave permission, and no students
were recruited from these. Subsequently, paid advertisements targeting university
students were used on Facebook. These advertisements appeared as electronic flyers on
the ‘Home Page’ of each Facebook user. The flyers targeted participants based on their
social network (e.g. university), age and gender.
Three paid flyers were used which had a web link and offered a £10
Amazon.co.uk voucher. Each flyer was for 1000 views by targeted Facebook university
students. These students were chosen by reviewing 184 UK Facebook university
communities. From this 104 UK Facebook university communities were selected based
on having: a) at least 300 members and b) universities with mathematical disciplines.
For each advertisement, 50 universities were chosen at random to proportionally
represent Scotland, Wales and England (the Northern Ireland university communities’
numbers were low). Using this process, there were only three responses from potential
participants of which only one participated. Additional posting on other internet student
forums did not help either. Two gatekeepers at the University of Reading and the
University of London posted advertisements on their electronic notice boards, but this
yielded no contact from any students.
There were perhaps several factors that influenced why students were not
coming through these avenues; firstly it was probably due to the computer equipment
requirements that were asked for such as web-cameras and microphones. Secondly, as
the study was hoping to use some type of advanced mathematics, it was necessary that
students had completed General Certificate of Secondary Education (GCSE)
Mathematics or equivalent and some kind of mathematical type subject at a higher level
either at A-level or at the university level for example in economics, statistics,
have a wide selection of students as possible, so it can be more generalisable to the
student population.
Dwyer, Hiltz and Passerini (2007) also recruited participants through Facebook
forums for their survey (they did not indicate how many forums); they were able to
recruit 69 Facebook participants in exchange for a music download. Thus, it is possible
for surveys to recruit participants via Facebook. Possibly asking for students to take part
in a study requiring them to provide a web camera and microphone reduced the chances
of finding students. Clough (2005) in her online survey of mobile learning had a high
response rate when she posted a link on an online mobile technology forum. It seems
plausible that if the survey is on a topic that the participants are enthusiastic about, then
the participants are more likely to respond to internet advertisements. Given that the
topic being advertised was mathematics, this maybe was not the best subject to find
enthusiasts to take part in the study. Even so, one participant (Participant 33) took part
because he was quite interested in how the Excel files were coded in VBA. The
researcher sent him a copy of the Excel file after his participation in the study.
3.6.2 Recruiting Students via Gatekeepers
However, to counteract these problems and accessing these students became an
issue. The next step was to find gatekeepers (Hammersley and Atkinson, 1995) that
could aid in finding these students through snow-balling techniques (Sapsford, 1999),
that is, by making contact through persons who knew undergraduate students such as
colleagues and friends. As the methods (via internet and snow-balling techniques) for
recruiting UK students failed considerably, a large portion of the students were recruited
from the University of the West Indies in Trinidad and Tobago. One gatekeeper was
able to access a large group of students. The choice of using remote observation as a
The use of gatekeepers at the University of the West Indies proved to be
successful. The past working relationship between the researcher and the gatekeepers
was instrumental in the finding and recommending of students. These students were
recruited personally rather than through advertisements. Further as a mini-laboratory
(for the remote observation) was set up by the gatekeeper with the required equipment
of a microphone and web camera, this made it easier to find students to take part in the
study as they did not have to provide their own equipment.
The gatekeepers only had a small pool of students whom they could personally
ask and who were willing to do mathematical tasks. For example, after the second
gatekeeper exhausted all her own students, the second gatekeeper found a third
gatekeeper to help provide the last few students (9) for the study. These were the
students who were mainly from engineering, mathematics and physics.
Further, the students recruited via the gatekeepers were remunerated with
chocolate bars at the end of the study, although they were not told in advance that they
would receive chocolate bars. This showed that remuneration was less important in
finding participants than having good gatekeepers who could personally ask students to
take part and find students who were comfortable with the subject area.
Additionally, as the students in Trinidad and Tobago had a similar secondary
school and tertiary system to the UK, this meant that results might be appropriate for
UK students as well. In Trinidad and Tobago as in the UK, there is a secondary school
examination that students take at age 16, which is called the Caribbean Examination
Council (CXC) examinations which are equivalent to the GCSE. Further, at age 18
students also need to matriculate through the UK General Certificate of Education
(GCE) at the A-level or a local A-level called the Caribbean Advanced Proficiency
Examination (CAPE, administered also by CXC) but the majority of the students take
UK. According to Association of Commonwealth Universities (2008), in Trinidad and
Tobago the university requires students to possess 2 GCE A-level subjects and 3 further
subjects at the CXC level. This is similar for the UK except where GCSE passes in the
subject levels are required. At the university level, the degree award system is the same
since the University of the West Indies was initially an external college of the
University of London until 1962 when it was granted charter to award its own degrees.