Pilot study

Pilot studies are considered important pre-procedure tools by researchers before the main study begins:

[T]he use of pilot studies as a context information management tool, implemented in the process of consolidating the research design, acted as in situ training for developing and testing the adequacy of data collection and analysis instruments, and as a relevance filter.

(Baptista Nunes, Tiago Martins, Zhou, Alajamy and Al Mamari, 2010, p. 83)

Testing and development of the data collection and analysis tools during a pilot study ensures that if necessary the research design can be modified and consolidated before the main study begins (Baptista Nunes et al., 2010). Therefore, prior to carrying out the main study a pilot study on a smaller scale was undertaken. The research site involved two teachers from two separate schools and one classroom observation in the second school. The participants and schools were not the same as those in the main study and

investigation time was shorter. The pilot study research questions were similar to those of the main study but were narrower in scope, i.e. they involved the three perspectives of context, teachers and pupils but only some of the dimensions (see Table 2.2) contained within these and the sub-questions (see Table 2.1):

1. Where English is integrated into the curriculum.

2. The type of teaching methods and strategies in use during the lessons. 3. Lesson planning.

4. Teaching aims. 5. Pupils’ perceptions.

6. Teacher training and influence on classroom practice. 7. Influence of government guidelines on teacher training.

The pilot study research questions investigated dimensions contained within sub-questions 2, 3 and 4, i.e. time factors, subject choices and themes for English, tools in use, and teacher competence and training. Using a picture questionnaire, sub-question 5

investigated pupils’ perceptions of English as a learning tool for diverse subjects. The data collected for the pilot study consisted of the following:

• Two individual teacher interviews: Teacher One (T1) and Teacher Two (T2).

• One classroom observation: (T2).

• Two individual pupil interviews: Pupil One (P1) and Pupil Two (P2).

• Two individual pupil picture questionnaires: as above (P1) and (P2).

• MoE policy document (2005).

Originally the first teacher (T1) had agreed to participate in the pilot study, but shortly before the study began she withdrew her consent, citing professional problems with both

the school and parents. Nonetheless, she did consent to participate in a semi-structured interview and this enabled some of the research questions to be addressed. In the second school, the pilot study was undertaken as planned. A semi-structured interview to investigate some of the dimensions contained within sub-questions 2, 3 and 4 was conducted with teacher (T2) prior to the classroom observation. One short post-study interview was conducted directly after the pilot classroom observation.

Audio-recording and field notes were in use during the interviews and observation.

Unfortunately, the audio-recording of the classroom practice was a complete failure due to technical problems with the Dictaphone. Therefore, observation data consisted of field notes and notes written from memory directly after the lesson. The observation and interview findings guided the development of the pupil picture questionnaire, which was issued the following week. This was necessary to ensure that the items pictured had relevance for the children regarding their knowledge and experience of English.

Nevertheless, it was necessary to simplify the mode in which the pupils could record their responses. Originally they were asked to choose which sticker from three types of sticker represented their perceptions of English use for learning diverse subjects. Unfortunately the children got confused with the signs on the stickers and could not remember what they represented. Therefore the picture questionnaire underwent revision and was reissued a week later, when it was easily completed by the pupils. Table 3.1 shows the sub-questions and dimensions investigated and the data collection methodology and strategies used for the analysis.

Table 3.1 Pilot study research questions and methodology

Participant Perspective Sub-

question Dimension Data collection tools Data analysis Teacher Two (T2) Teacher 2 Tools, subject choice Semi- structured interview and classroom observation Qualitative T2 Teacher 3 Teacher attitude, teacher training Semi- structured interview and classroom observation Qualitative T2 Teacher 4 Enjoyment/ Interest Semi- structured interview and classroom observation Qualitative T2 Teacher 2, 3, 4 Teaching style, interactions, tools, enjoyment Classroom observation Qualitative

Two pupils Pupils 5 English for

learning subjects Picture questionnaire Quantitative Teacher One (T1)

Teacher 2, 3, 4 Same as case

study (pilot) Teacher (T2) Semi- structured interview Qualitative MoE document (2005) Contextual 1, 2, 3, 4 Teaching policy and guidelines Document Qualitative

In light of the findings of the pilot study, which was originally set in the framework of a qualitative research design, a sharpening of the main study focus and reflection and

analysis of the methodology and strategies in use to investigate the main research question was undertaken. Problems identified in the pilot study, i.e. pupil picture questionnaire and technical problems, were addressed and resolved. The data collection tools and strategies used in the pilot research study could all be retained. However, the study design was expanded and developed from a qualitative study into a mixed methods study that not only enabled the investigation of a wider population, which might yield generalisations, but also offered the opportunity to increase rich description (Geertz, 1973) in the findings.

Expanding the study involved the addition of a quantitative element (teacher questionnaire) entailing some statistical analysis. It was also necessary to pilot the questionnaire (Dörnyei and Taguchi, 2010) prior to the main study.

Contextual features were also developed further to include data collection from the teacher training universities (PHs) to identify links between teacher training at one level and teacher practice in the actual classroom (Table 2.1: sub-questions 6 and 7; Table 2.2). The findings could then be analysed in relation to MoE (2005, 2012) policy guidelines. The interviews with academics from the PH provided a deeper insight into how the training on offer provides future teachers with the necessary FL skills and competence in their

classrooms. Associations between responses from the teacher questionnaire and teacher training could then be analysed. The pupil picture questionnaire indicated that connections could be made between teacher training and teaching practice involving pupils’

perceptions of English with regard to the English content and subjects. However, a deeper insight into pupils’ perceptions of teaching practice and learning during English was strived for in order to provide comparative data between teachers’ and pupils’ perceptions of the English teaching/learning taking place in the classroom. Associations between the educational aims and the teaching guidelines of the MoE (2005) in combination with the actual teaching taking place in the classroom could then be made. Therefore, the

development of an original research tool – pupils’ drawings – to investigate sub-question 5 (pupils’ perceptions) in more detail was also made after the initial study and piloted.

Making modifications to the main study post initial study is a process which reflects the characteristics of qualitative research design as perceived in the five-stage interactive model identified by Maxwell and Loomis (2003). Maxwell (1996, pp. 1–8) refers to this as an:

‘interactive’ approach, whereby a qualitative study’s purpose, research questions, conceptual context, methods, and concern for validity all continually interact.

(cited in Yin, 2011, p. 77; original italics)

Yin (2011) describes how the importance of research design prior to data collection in qualitative studies has not always been clearly recognised by researchers, stating that ‘the design is a recursive one’ (p. 77). Furthermore, he explains how adaptations can and are continually made in a qualitative study.

The necessity to reflect upon researcher insider–outsider status (Hellawell, 2006) with the participants is also important, and as an English-born primary teacher in Austria with personal experience of the classroom context, I am aware that researcher ‘blindness’ to familiar events may influence critical observation of the phenomena.

Simmel(1950, p. 402) argues that an outsider:

can give the stranger’s perceptions and judgements a particular kind of objectivity not usually granted to the insider.

(cited in Hellawell, 2006, p. 486)

However, Schütz (1964) argues that insider knowledge may remain hidden to outside observers. In view of this my status in the case study schools as an unknown outside observer/researcher with insider knowledge and familiarity of the primary school context complemented one another during the classroom observations.