Chapter 4 Research Design
4.2. Designing the evaluation research
4.2.1. Sampling
A sample frame is formulated by identifying the target population and deciding on a strategy of how to choose participants. The resulting sample needs to be representative of the target population, authoritative, knowledgeable, credible, as well as accessible, of a reasonable size, and fits with the overall research design (Newby, 2014; Cohen, Manion and Morrison, 2011; Gray, 2004).
Since this research is an attempt to formulate a framework for instructional design of learning experiences, the data that need to be collected should lead to evaluating and refining the framework as well as to evaluating and informing the instructional design itself. Therefore, the pertinent target audience for these types of data is the lecturers who design the learning experience.
Getting their perspective on the assignment design and on the implementation of the design will help understand how designs are made. It also leads to some understanding of the causal relationships between different elements of the design and any reported successes or shortcomings. This, in turn, can inform the FUAD framework in the sense of refining, removing or adding elements that are not in the framework initially. It also tests whether the FUAD framework can be used as a lens and a diagnostic tool.
For the above-mentioned purpose, a non-probability, purposive sample, also known as a judgment sample (Miller and Salkind, 2002), was used to identify lecturers who had designed TEL experiences. According to Teddlie and Yu (2007), a purposive sample is used to achieve representativeness, enable comparison, focus on unique issues, and can lead to the generation of theory
or broadly defined themes. In a purposive sample, the researcher uses her own judgment in identifying respondent lecturers according to pre-set criteria (Burton, Brundrett and Jones, 2008) that form the sample frame. In the case of this research, there are three main pre-set criteria for inviting participants to share their assignment designs (Figure 4.2). These are: 1. assignment features in terms of targeting different assignment products from a variety of disciplines, specific assignment features, type of work (individual or group work), and weighting of assignment or project; 2. different levels of study: tertiary, undergraduate, and post-graduate; and 3. context: different HE institutions in different countries.
Figure4.2. Sampling frame
The sampling strategy adopted for this research follows seven of Cohen, Manion and Morrison’s (2011) eight stages for planning a sampling strategy (Table 4.1). A non-probability, purposive sample of 16 assignments was chosen and the lecturers who designed them were invited for an interview. Access to lecturers and assignments was possible due to the fact that they were part of the researcher’s professional network. Some were approached personally, others through email or LinkedIn. Stage 8 was not applicable as
there was no need to adjust data. Data were qualitative, and all details were embraced as enriching insights to the research.
Table 4.1. Stages of planning the sampling strategies
The choice of lecturers to interview was initially based on the above- mentioned criteria of assignment features, level of study and context (institutional and geographical). Having set these criteria, the first round of invitations for interviews was sent to colleagues at my work place. I
approached nine lecturers, three because they were known for their innovative approach in teaching (assignments 5.1.1. smart object prototype, 5.4.1. App
design, and 5.7.1. PID) and two because they taught post-graduate students (assignments 5.2.1. mini conference presentation and 5.7.2. private cloud platform), and the remaining four were approached because they were in different departments (the business school: assignment 5.8.1. taxation coursework; foundation programme: assignment 5.6.1. sustainability leaflet; and two from the media department). However, the latter two from the media department only shared documents and did not give an interview; therefore, they were excluded. For the second round, I contacted lecturers from my professional network of colleagues whom I have previously worked with or met at conferences and academic events. Invitations for interviews were sent to colleagues in different countries based on their geographical location, to ensure that the sample was more international; two from Canada:
assignments 5.1.2. DAL project and 5.8.2. logical database design; one from Egypt: assignment 5.4.2. Arabic language assessment; and one from Oman and one from the United States who only shared documents but did not give an interview, and therefore were excluded. Four lecturers were contacted due to their affiliation with government-funded institutes in the United Arab
Emirates (assignments 5.2.2. vocabulary video, 5.3.1. lesson plan, 5.6.2. reflective journal, and 5.6.3. case study presentation).
Contacting lecturers from different contexts (educational and geographical) was deliberate, and resulted in achieving a variety of assignment types in terms of final assignment product. More specifically, I approached three lecturers particularly because I was aware that their assignments included special features that could inform the FUAD framework in different ways. The mini-conference assignment (5.2.1) was selected purposefully, because it
featured inviting students to co-construct success criteria. Therefore,
understanding the lecturer’s perspective on why she chose to do that would inform the argument for FUAD principle 2: Co-construction of success criteria. Similarly, the sustainability leaflet assignment (5.6.1) was chosen as I was aware that students struggled to complete it due to their lack of digital literacy skills. Hence, the lecturer’s perspective could inform detail for FUAD 6: accessibility to technology and skills support. The listening lesson plan assignment (5.3.1) was chosen because of the type of assignment and how it required students to present a lesson to their peers, coupled with the type of freedom the instructions of the assignment allowed. This would inform FUAD principle 3: generic assignment description.
The resulting sample (see Table 4.2) consisted of sixteen assignments from six different universities in four different countries. The assignments were from ten different departments and varied between individual work to group work, formative to summative, from heavily-weighted to bonus grade. Assignment types included coursework, multi-product assignments, hands-on
implementations, leaflet, poster, video, essay, reflection, case study and lesson plan.
Table 4.2. Sample details
This sample was large enough to generate thick descriptions and to reach saturation but not too large to cause data overload or move towards
generalisability (Onwuegbuzie and Leech, 2005). By saturation, I mean that data analysis was no longer giving new ideas and started to repeat and confirm data sets.