Collecting Data from Research Participants

As in the Colby-Kelly and Turner (2007) study, the current study used interviews to collect data from the teachers. Additionally, the current study incorporated a focus group discussion and a delayed questionnaire. As discussed earlier, all research tools were developed for the purpose of this study. The subsections below report the rationale for

7 _{Record of Work Done (ROWD) is a name used for the documents by the school. For the description of ROWDs see}

109

selecting each research method and the purpose for deploying it as well as the processes of developing the research tools.

3.2.4.1.1 Teacher Interviews

Interview as a research method enables participants ‘to discuss their interpretations of the world in which they live and to express how they regard situations from their point of view’ (Cohen et al., 2007, p. 349). For that reasons, interviews were identified as a useful research method to investigate the teachers’ own understanding of AfL. This choice was similar to other studies which have investigated teachers’ beliefs about self- and peer- assessment (Dixon, Hawe & Parr, 2011) or teachers’ interpretations of the effectiveness of assessment (Butler & Lee, 2010) and the implementation of AfL (Colby-Kelly & Turner, 2007; Lee & Coniam, 2013, Rea-Dickins, 2001) and assessment in TEYL contexts (Edelenbos & Kubanek-German, 2004). Table 3.7 below summarises studies which used teacher interviews for purposes similar to those in the current study.

110

Table 3.7: Studies which used teacher interviews

Study Focus of the study Focus of teacher interviews

Edelenbos and Kubanek-German, 2004 Classroom assessment of young language learners

To investigate ‘diagnostic activities’ (p. 264) used by YL teachers

Rea-Dickins, 2001 Formative assessment in an EAL context

To ‘provide evidence for a range of strategies in relation to the

implementation of classroom assessment and, in particular, formative assessment’ (p. 432) Lee and Coniam,

2013

Assessment for

Learning in Secondary School L2 (English)

To investigate ‘the implementation of AfL in writing’ (p. 38)

Butler and Lee, 2010

Self-assessment of L2 (English) in a Primary School

To ‘understand the teachers’

observations and perceptions towards the effectiveness of self-assessment and to elicit the teachers’ insights regarding how self-assessment may be implemented effectively in their respective teaching

environments.’(p. 14) Dixon, Hawe and

Parr, 2011 and Hawe and Dixon, 2014

‘Teachers’ espoused beliefs about self- and peer- assessment and their congruence with practice; (p. 365) in New Zealand state schools

To investigate teachers’ beliefs about learners conducting peer and self- assessment Colby-Kelly and Turner, 2007 Assessment for Learning in an EAP context

To investigate teachers’ views on using AfL in speaking activities

Deploying the interview as a research method had some limitations. First, data which were collected were teacher’s subjective interpretations of classroom realities. Hence, that data provided insights into teachers’ understanding about the implementation and impact of AfL. Secondly, the one-to-one nature of the interviews may have posed a risk of some teachers saying what they thought was expected of them rather than sharing their beliefs. To minimise that risk, as the researcher I ensured that I built a positive relationship with each teacher before the interviews were scheduled. It is believed that this contributed to teachers feeling more relaxed and willing to share their honest opinions with the researcher during the interviews. Moreover, at the beginning of each interview, I briefed

111

each interviewee about their anonymity and the purposes of the interviews, drawing attention to the fact that the research was focusing on gathering data about the teachers’ understanding of AfL and was not testing their knowledge about it (Appendix 9). Finally, if teachers were not able to readily respond to the prompts included in the schedule, allowances were made for them to come back to any topic at a later time during the interview.

The process of designing a protocol entailed a number of steps specified by Cohen et al. (2007). First, the purpose for conducting interviews was defined. In the current study, the purpose was to gather information with a direct bearing on the research questions (Tuckman & Harper, 2012). This purpose was then translated into the specific aims of the interviews, which was to collect data illustrating the teachers’ beliefs about:

 what AfL means in a TEYL context;

 how they implemented AfL in TEYL classrooms;

 the impact of using AfL on interactions within TEYL classrooms.

The next steps involved designing a semi-structured schedule using a standardised open- ended approach (Patton, 1990) by summarising the outcomes of the literature review, the preliminary outcomes of lesson observations from the pilot study (n=4) and mapping them out against the interview objectives. This resulted in preparing thirteen open-ended questions. The wording of the prompts took into account the profiles of the teachers. The interviews took place between January and March 2012. The procedures used to analyse the data obtained through the interviews are discussed in Section 3.3.1. The following section reports on how the focus group schedule was developed.

3.2.4.1.2 Focus Groups

Focus groups are a useful research method which can be used to triangulate findings from other sources (Somekh & Lewin, 2005). In the current study, a focus group with eight teachers was organised to verify the draft findings from individual interviews. The aim was to validate the researcher’s interpretations of the interview data and to gain more in- depth insights into the teachers’ understanding of AfL. Moreover, the purpose of using a focus group as a research method was to elicit collective (not individual) beliefs through the data that emerged from the interactions within the group (Morgan, 1988).

112

As Cohen et al. (2007) rightly notice, despite the claims that ‘its potential is considerable, the focus group, as a particular kind of group interviewing, still has to find its way into educational circles to the extent that it has in other areas of life’ (p. 377). Adopting the focus group as a method to verify and expand findings from individual interviews contributed to the innovative nature of this study. It was influenced by Lee and Coniam’s (2013) study that used this method with adolescents to investigate learners’ perspectives on how implementing AfL impacted on their motivation and writing performance.

Prior to data collection, the researcher had been trained in how to conduct focus groups and was therefore aware of the challenges involved in facilitating a focus group, such as managing the group dynamics in order to ensure participation and the coverage of relevant topics.

The schedule for the focus group (Appendix 10) was developed through a process similar to that adopted for developing the interview schedule. The formulation of the questions was informed by draft findings from the analysis of the interview transcripts. The focus group discussion with all eight teachers took place in May 2012. The process of the analysis of the data collected through the focus group is reported in Section 3.3.1.

3.2.4.1.3 Teacher Questionnaire

Questionnaires as a research tool are relatively easy and time efficient to administer and allow for collecting quantitative as well as qualitative data (Somekh & Lewin 2005). Questionnaires are often used in educational research: most often to investigate attitudes, motivation and perceptions. Studies of AfL, which deployed questionnaires for such purposes, including those based in TEYL contexts, collected data from learners (Butler & Lee, 2006; Enever, 2011; Gattullo, 2000, Lee & Coniam, 2013,) and teachers (Colby- Kelly & Turner, 2007, Rea-Dickins & Gardner, 2000). Table 3.8 below provides details of the purposes and designs of questionnaires used to collect data from teachers. As evident from Table 3.8, in studies of classroom assessment, questionnaires were deployed with small samples of teachers, similar to the size of the sample in the current study.

113

Table 3.8: Studies about AfL, which gather data from teachers through questionnaires

Study Focus of the

questionnaire Number of respondents Design Colby-Kelly & Turner, 2007 ‘Teachers’ perceptions about classroom based assessment’ (p.19)

9 4 point Likert-type scale

Rea-Dickins & Gardner, 2000

Teachers’ perceptions of the assessment of EAL students

9 Open-ended

In the current study, a questionnaire was selected as a method which allowed obtaining a longitudinal perspective and ensuring time triangulation. A delayed teacher questionnaire (Appendix 13) was administered sixteen months after the cross-sectional data collection stage ended. The aim of the questionnaire was to gather data about changes in the implementation of AfL over time.

The design incorporated a Likert-type scale and open-ended questions. The scale, ranging from 1 (never) or (almost never) to 5 (every lesson) or (almost every lesson), was deployed. A list of all AfL techniques identified in the data thus far was provided. The teachers were asked to indicate how often they used each of the techniques. Additionally, space was provided for recording techniques that teachers used but which had not been included in the list. This was followed by a number of open-ended questions which aimed to gain more nuanced insights into the developments that occurred over time. The delayed questionnaire was piloted in September 2013 and administered in October 2013.

This section has discussed the research methods used for collecting data from teachers. The rationale for selecting an interview, a focus group and a delayed questionnaire as data collection methods has been provided. This section has also reported how the respective research tools were developed. The next section reports on how the lesson observation schedule was developed.