3.4 Research design
3.5.1 Exploratory phase
During this phase, collaborative talk in authentic workplace settings was explored in order to inform the development of teaching materials during the second, development phase of the research design.
To gain an insight into how collaborative talk occurs in authentic workplace settings, observations were initially conducted of participants talking in collaborative scenarios. Observations were conducted with the intention that the teaching materials could be informed by in-depth, rich accounts of the particulars of real interactions (Griffiths & Macleod, 2008).
However, capturing genuine collaborative talk proved difficult, particularly given that access to ‘collaborative scenarios’ largely involved observing agenda-led meetings. Genuine collaborative talk often occurs informally (Asmu & Svenning, 2009) and may be better captured through ethnographic methods which were considered impractical and unnecessary for the main purposes of this study.
Though observations revealed some examples of language use unique to collaborative workplace contexts, interviews proved more useful and economical. Employers and employees, sometimes those previously observed, were asked for their perspectives on the talk skills required for collaboration and asked to describe authentic collaborative scenarios which could inform task design.
Throughout this phase, several workplace settings were visited. Time was spent with The Communication Trust and Dialogics Ltd, who were able to provide a wealth of
111
specialist knowledge, as well as further opportunities to observe or discuss collaborative talk within workplace settings.
3.5.2 Development Phase
Informed by the exploratory phase, a teaching unit for collaborative talk was devised in preparation for implementation. This phase involved working in collaboration with the participating teachers. Resources were trialed by both teachers and researcher, then reviewed and amended.
Throughout this phase, frequent meetings were arranged with the participating teachers, providing an opportunity to discuss resources but also the theoretical underpinnings of the research. By recording the meetings, teachers’ perspectives expressed in an informal and unstructured context were captured.
This phase also provided an opportunity to observe informally the implementation classes. This allowed the researcher to become familiar with class routines and put names to faces, while also reducing reactivity effects which may result from the presence of an unfamiliar adult (Cohen, Manion & Morrison, 2007). Teachers were also encouraged to use audio recorders in activities to ease potential anxieties ahead of implementation. Field notes written during these observations detailed classroom arrangements, interactions between teacher and students, and so forth.
3.5.3 Implementation Phase
This phase used the opportunities provided by the GCSE Speaking & Listening requirements for English GCSE to engage in and be assessed upon group discussion and interaction skills as the curriculum focus for investigation (Edexcel, 2010).
112
The teaching unit was implemented over a three week period, first in one school, then in the other. This period represented an intense period of data collection, during which video and audio recorders captured teachers’ implementation of the teaching unit and students’ developing use and understanding of collaborative talk processes. The teacher wore an audio recorder in each lesson so that her interactions during students’ independent work could be captured, as well as whole-class interactions. An audio recorder was placed on each group’s table and a camera was positioned to capture each group. The video and audio recordings were later synchronized. This approach facilitated different perspectives on the teacher’s role and captured students’ ‘public’ as well as ‘private’ group talk.
Teachers conducted a pre-implementation Speaking & Listening assessment and produced qualitative information for individual students and the class. The teaching unit would involve conducting an assessment which would serve as a post- implementation assessment, at which point teachers were asked to write further qualitative comments.
Groups were interviewed at intervals throughout the project. The consistency and frequency of these interviews was somewhat dependent on the timetable: it was easier to interview groups in School 1 because English lessons frequently preceded break or lunchtimes, for instance. The interview data served to complement the observational data and inform interpretations where appropriate.
Each student was allocated a booklet within which they would write individual and group responses to questions or tasks and complete self-evaluations. The teaching approach placed particular emphasis on gathering students’ perceptions of their progress and development via meta-linguistic and meta-cognitive reflections. These
113
booklets provided another perspective, allowing students to share ‘in private’, possibly making points which conflicted with the group.
After implementation, meetings were held with the teachers in which the data was discussed, as was the implementation process and implications for teaching and further research.
114 Chart 3.1: Three phase research design
Exploratory Observations Interviews Development of Materials Development: trial, evaluate, refine teaching materials
with teachers and students
Implementation
SCHOOL 1115
Timescale Research Activity
Iterative analysis and development of materials Phase 1: Exploratory February - May 2011 Workplace observations Workplace interviews Written reflections May – July 2011
Development of teaching materials
Meetings with participating teachers and school visits Written reflections
Phase 2: Development
July - October 2011
Further development of teaching materials Pre-implementation teacher interviews
Task trials with two classes Students’ evaluations gathered
Phase 3: Implementation
November -
December 2011
Pre-implementation student assessment data Implement SoW: School 1 and 2
Video capture Group interviews Gather student booklets
Post-implementation student assessment data January – February
2012
Consolidation of teaching materials Post-implementation teacher meetings Table 3.1: Research design timescale
3.6 Access and Participants
3.6.1 Workplace
After establishing a contact, workplace settings were invited to participate by e-mail. The e-mail gave a brief outline of the research and its aims and described the type of talk which I was hoping to observe (appendix B). Written consent was sought from each interview participant (appendix C).
116
With help from The Communication Trust, consortium members such as The Children’s Society were approached by e-mail or telephone. Visits to a number of these settings were conducted, providing an opportunity to glean expertise but also observe collaborative talk in the workplace. As a significant developer of collaborative talk resources, though not a Communication Trust consortium member, Dialogics Ltd were approached by e-mail.
3.6.2 Teachers and Schools
Secondary state schools in the South West participated in the study. I met Abigail (Teacher 1) during the National Association for the Teaching of English (NATE) conference in 2011. Abigail and I discussed the project and she signaled her interest in participating. I approached the English department where I once worked, and Vicky (Teacher 2), also signaled interest in participating. Once interest in participation was expressed further details of the research was given, particularly regarding the commitment required (appendix D). After this, contact was made with Head Teachers and Heads of Department and a Memorandum of Understanding was completed (appendix E). Consent was then secured from individual teachers (appendix F).
3.6.3 Students
The research information provided to teachers, as well as the Memorandum of Understanding, outlined the requirement for GCSE classes to participate. However, late into the development phase, some unanticipated changes were made. The timescale probably resulted in this: schools and teacher participants were secured ahead of groups being arranged for the new academic year, when implementation would take place (though difficulties may have arisen regardless). In School 1, the
117
Head of Department made the unforeseen decision to arrange the top ability Year 10 students into groups by gender. So, Abigail’s group would be a top-set all-girls group. In School 2, Vicky’s responsibilities changed, and as a result of various pressures, she expressed a preference for implementing the teaching unit with a mixed ability, mixed gender Year 9 class. Participants therefore formed a convenience sample; classes included one group of 32 Year 10 students in School 1 and one group of 28 Year 9 students in School 2.
Given the focus on understanding the particulars of collaboration, the variation in groups was not considered problematic; in fact, the research design allowed for this flexibility. If the study’s aim is to develop practical, useful teaching strategies, then their application to a broad range of students must be considered. For the same reason, students with SEN and EAL were not excluded from the study; School 2’s class included a student with EAL (English as a second language) and one with EABD (emotional and behavioural difficulties). Both classes still represented ‘normal’ state school classes, representing a range of abilities.
This approach focused the research on the development of students’ skills within familiar, ‘real’ contexts which were relevant to them, not on the experimentation of theories in contrived structured groups or settings. Forming a contrived sample would have undermined this aim and the collaborative nature of the study. Although selecting students in advance for certain features may allow a first approach to comparison (Flick, 2007), arranging students by characteristics may undermine the perspective of students as complex social beings. It may also be inappropriate to sample populations by attributes because of how these attributes may be defined in the research (Silverman, 2005).
118
School 1: Bayside College School 2: Spring Lane College
Teacher 1: Abigail Teacher 2: Vicky
Year 10 (14-15) all-girls high ability class Year 9 (13-14) mixed gender and ability class
32 students in total 28 students in total
8 groups of 4 7 groups of 4
Table 3.2: Participants
3.6.4 Students’ Informed Consent
Ahead of the implementation phase, students were informed of the project and its aims through a presentation given by the researcher (appendix G), providing opportunities for students to ask questions. Students were given a letter and consent form to fill in during the lesson and given another letter and consent form for parents (appendix H, I). While students filled out consent forms I circulated the room to check understanding and allow students to ask further questions. Students were asked to fill in forms during the lesson but in sending letters home, students were given the opportunity to discuss any concerns with parents who could have objected to their participation. All parents returned consent forms, having agreed to students’ participation.
Because the priority was to secure students as participants, the first consent was not concerned with the use of the data. At the end of the project, students were asked to indicate their preference for the use of the data (appendix J). They could signal their consent for the data to be used in research presentations and in resources or for CPD purposes. It was later decided that this form would not be taken as evidence of students’ informed consent for videos to be used as resources or CPD materials.
119
To recognize and celebrate their participation in the project, students were presented with a certificate and letter of appreciation (appendix K). Students were encouraged to refer to the experience on their CVs.
Data Collection Instrument
Phase 1: Exploratory Workplace observations Workplace interviews Unstructured/ semi- structured observation schedule Semi-structured interview schedule Phase 2: Development Teacher meetings
Student reflections and evaluations
Teachers’ critical reflections and evaluations of the materials
Informal observations of implementation groups
Meeting agendas; audio recorder
Task evaluation pro-forma
Informal observation pro- forma
Phase 3: Implementation
Pre-
implementation
Pre- project qualitative and quantitative student attainment data
Pre-implementation assessment pro-forma
Implementation Video and audio data of lessons Reflections on progress and development Group interviews Teaching materials
Video and audio recorder
Student booklets Semi-structured interview schedule Post- implementation Post-project qualitative and quantitative student attainment data Teacher meetings Teachers’ case descriptions Post- implementation assessment pro-forma
Meeting agendas; audio recorder
Case descriptions form
120 3.7 Methods of Data Collection
3.7.1 Phase 1: Exploratory
Exploratory Phase: Observations
During the first phase, data collected during observations in the workplace ranged from detailed notation to holistic descriptions of events and behaviour. An assumption of this approach is that a profound understanding of the world can be gained through observations in natural settings (Anderson, 1998). The observations were unobtrusive, but it is acknowledged that the presence of an observer and an ‘outsider’ may affect the way people interact. These unstructured observations informed a later semi-structured observation schedule (appendix L).
Exploratory Phase: Interviews
Following observations in the workplace, semi-structured interviews were conducted with employers and employees to establish their perspectives on the skills required for effective collaborative talk (appendix M). Interviewees were also invited to describe authentic collaborative workplace scenarios which might inform task design. This method reflects the epistemological position that to understand the meaning- making process we should start from reconstructing how people, institutions and communications construct worlds and social realities, to show how meanings are built up in interactive processes (Flick, 2007).
Interviews made it possible to invite comments on the collaborative talk previously observed. Although this aims to capture authentic insights into participants’ meanings (Silverman, 1993), it is important to recognise the difficulty in capturing the meanings participants attribute to words previously spoken. Detailed field notes were
121
written to capture features of the workplace settings and other details perceived as relevant at the time. My own reactions and thoughts post-interview were written immediately and limitations considered, my own characteristics having helped or hindered the interview process (Oppenheim, 1992).
3.7.2 Phase 2: Development
Development Phase: Teacher Meetings
At intervals throughout the development phase, meetings were held with the participating teachers, either in School 2 or at the researcher’s home. Agendas were devised for these meetings and records maintained, including opportunities for the teachers to discuss the teaching unit and underpinning theory (appendix N), express concerns or reflect on teaching trials. This provided a means of capturing teachers’ insights but also any developments from meeting to meeting. By recording meetings routinely, it was possible to capture teachers’ views in a context less formal than via the interview. Prompted by an agenda topic or question, teachers would also talk together for prolonged periods, uninterrupted by the researcher. During these meetings it was possible to collect teachers’ written descriptions of their teaching history, relationships with students and so forth, to inform later analysis.
Development Phase: Trials
Teachers were provided with a draft teaching unit with accompanying resources, which were discussed during meetings. During this phase, activities from the teaching unit were trialed with a variety of classes at Key Stages 3 & 4, as convenient. These were discussed during meetings. Both teachers were able to trial several tasks as ‘stand-alone’ activities. During this period, I did some supply
122
teaching in School 2 when I was also able to deliver a sequence of trial lessons to a year 9 class. Students were invited to comment on the tasks during lessons or asked to complete written evaluations (appendix O).
Development Phase: Informal Observations
The implementation groups were observed in a small number of lessons prior to implementation. This allowed classes to get used to the researcher’s presence in the classroom. This process allowed the researcher to consider the relationships between teacher and students, and between students. Observations of their interactions informed the guidance that was given during the final pre- implementation teacher meeting day. Again, field notes were written to document observations.
3.7.3 Phase 3: Implementation
During the implementation phase, the ‘core’ data collection comprised the audio and video data and was complemented by ‘additional’ data collected through a variety of methods.
Teacher Audio Audio collected via recorder worn by teachers
2 teachers 18 hours total
Group Video Cameras captured each group 8 groups school 1; 7 groups school 2
135 hours total Group Audio Audio collected via recorder placed
on each group table Table 3.4: Core data set
123
Assessment Data Pre and post GCSE Interacting and Responding assessment
60 students Group Interviews Group interviews conducted where
possible during SoW
14 interviews school 1 5 interviews school 2 Booklets Booklets maintained throughout
duration of SoW
60 booklets Teacher Meeting
Recording
Trial, pre and post-implementation teacher meeting recordings
10 recordings Table 3.5: Additional data set
Implementation Phase: Video and Audio Data
Video and audio recordings of the lessons provided the rich core data set of talk and interaction (Parakyla, 2005), preserving the visual aspect of talk. The methods of data collection, crucially, ensured observational data of group talk independent of the teacher in the naturalistic classroom. By recording the teacher separately, it was possible to capture whole-class interactions and interactions or interventions with individual groups or students during their independent work.
Lessons were recorded in their entirety, as they occurred in the naturalistic setting. This approach was in keeping with the emergent design, and recognized that students’ collaborative activity takes place within a broad context, framed by preceding and subsequent teacher instruction. By examining students’ developments and perspectives over time, the long-term trajectory of the process of teaching and learning was appreciated, recognising that these processes cannot be understood as discrete educational events (Mercer, 2008).
Implementation Phase: Assessment Data
Although the research design emphasizes the collection of observational data, individual assessment data (qualitative and quantitative) was gathered to complement the core data set. Though the notion of ‘abilities or skills as stable
124
possessions of individuals’ (Rogoff, Radziszewska & Masiello, 1995, p. 144) is indeed questionable, individual accounts of learning may nevertheless be useful (Sfard, 1998; Packer & Goicoechea, 2000; Cobb, 1994). Eliciting individual assessments for collaborative activity is indeed a contradication; however, as it stands, this is a process required by English GCSE (and throughout our competitive education system). And while this study sought to promote the place of Speaking & Listening, it was imperative that the data collection fit with schools’ curriculum and assessment procedures.
Teachers were asked to complete a pre-implementation Speaking & Listening assessment and to write qualitative information alongside. Teachers were also asked to provide current GCSE or Key Stage 3 levels and predicted grades for GCSE English. This revealed any discrepancies between ‘general’ English ability and Speaking & Listening ‘ability’. Teachers were asked to note relevant information about the student: strengths and weaknesses, SEN or EAL, relationships, and so forth. In the final lessons, teachers circulated the room and listened to groups’ talk to elicit post-implementation assessments for Speaking & Listening. Alongside GCSE grades, the teacher wrote qualitative comments for students in the back of their booklets. Teachers also wrote a comment on each group’s progress as a whole. This was complemented by students’ post-implementation self-evaluative comments.
Gathering the assessment data was more a priority for teachers than for the research. The research is more interested in examining students’ interactions and their development throughout the duration of the teaching unit. However, the assessments provoked some interesting discussion regarding the complexities of ‘marking’ in this area and nevertheless, contributed another means of conceptualizing students’ development. It also revealed teachers’ perceptions of
125
students’ communicative skills, sometimes exposing discrepancies with what was observed in the data. Quite simply, the assessment data provided another ‘angle’ on the data collection while also serving the needs of teachers and students.
Implementation Phase: Group Interviews
By interviewing students in their implementation groups, students were able to talk with each other, referring to a shared history and minimizing the role and influence of the researcher. The interviews during this phase were focused on gathering students’ reflections on their progress and understanding (appendix P). Consistent with an interpretive approach, questions avoided being so specific, that other avenues of inquiry were shut off (Bryman, 2008), allowing enough flexibility for the respondents to shape and frame the discussion (Marshall & Rossman, 1999. The interviews were regarded as a socially situated event, where data must be interpreted against the background of the context in which they were produced (Hammersley & Atkinson, 2007).
The use of group interviews is grounded in the conception of knowledge as generated between humans, emphasising the social situatedness of research data (Cohen, Manion & Morrison, 2007). Adolescents may prefer to express their views in the private of an individual interview, or may prefer the social security of a group