Changes emerging from the pilot study

The changes emerging from the pilot study are explained in this subsection by referring back to the methods in the order they are presented above. Whether a method was dropped or was kept for the main study, the reason or reasons for the choice made are provided.

Six changes emerged from the pilot study. The first change regards the questionnaire for learners. The questionnaire for learners was dropped in the main study as it was found to contain information that was not useful, as seen in the first three questions. In the first question, for example, information regarding ages was found to be unnecessary as participants were adult workplace professionals; they are responsible for themselves. The second question was not necessary because English is the language of work in the target workplace. That is, the fact that the Angolan national employees will continue improving it can be taken for granted. A similar point can be made with regard to the third question. That is, whether participants recognised the need to learn English or not, they would learn it as it is the requirement of the target workplace.

Other questions were illuminating as directly or indirectly they sought data about writing. For example, the frequency of the participants who did office

97

work was approximately 81 per cent and the frequency with which the participants used the language skills (listening, speaking, reading and writing) indicated that reading and writing corresponded to approximately 95 per cent. However, I felt that, for the purpose of genre analysis, texts that the target learners had written would be sufficient and more illuminating as they would provide insights into what the target learners did well and where they might need support.

The second emerging change concerns the texts written by the target learners. These texts were still used in the main study, but they were collected again for the purpose of ‘triangulation’ (Bryman, 2008, p. 607; OpenLearn, 2014). That is to say, text collection was repetitively used to ensure the texts collected are the ones most frequently written in the target workplace. The change here resides in the fact that the number of texts increased in the main study. More specifically, 114 texts were collected in the main study while only 32 texts were collected in the pilot study.

The third change that emerged from the pilot study is about the texts written by experts. Texts written by experts were dropped in that those experts were not part of the target workplace. As was stated above, the texts were gleaned from textbooks designed for the purpose of teaching business writing in different institutions (Hogan, 2005; Poe, 2006; Brieger, 2011). While such textbooks might give general views of writing in business, they may not be sufficient in that each business institution could be considered unique.

98

The fourth change regards the experiment. The experiment was still used, but it changed in terms of the number of participants, focus, and assessment of learners’ achievement. While the groups in the pilot study were composed of 10 participants each, each group in the main study was composed of 18 participants. This change was made in light of the feedback I received on the Year 1 Final Report and my further reading. Similar to the feedback I received on the Year 1 Final Report, Dörnyei (2007, p. 99) suggested that a ‘comparative and experimental’ study should have a minimum of 15 participants. Thus, the number of participants in the main study was raised to 18 per each group.

Regarding focus, in the pilot study, attention was equally focused on the effects of explicit and implicit genre-based approaches, but in the main study main attention was focused on the effect of an explicit genre-based approach. However, the implicit genre-based approach was still used as control of the explicit genre-based approach. In other words, I had a control group of learners who did not receive genre instructions in the classroom, but they were engaged on the job. More clarification on the control group will be provided later in the present chapter.

The focus changed for two reasons. The first reason was that I had to balance between my work and research commitments, as I did in the pilot study. That is, working with both experimental and the control groups in the classroom would have taken most of my work time. The second reason was an attempt to control factors that possibly distorted the findings in the pilot study. For example, although I provided my co-worker

99

with a lesson plan to teach the implicit-genre group in the pilot study (Appendix 3), I found it hard to monitor what happened in my co-worker’s classroom. More clearly, it might have happened that my co-worker directly or indirectly explicitly focused on the genre features.

Regarding assessment of learners’ achievements, in the pilot study a focus was only made on the written products in the pre-test and post-test. In light of my further reading of previous researchers’ studies on explicit genre-based approaches such as Cheng (2007) and Yayli (2011), I decided to add a component that engaged learners in the lessons during the experiment (the engagement is explained later in the present chapter). Furthermore, in the pilot study learners’ written products were assessed through a ‘texture index’ by Roseberry (1995, p. 205). While Roseberry (1995) considered such a formula powerful in measuring readability in education and business contexts, I realised that the formula was initially used with long expository academic texts. As the present study dealt with short email messages (requests), Roseberry’s formula was dropped.

The fifth change emerging from the pilot study appertains to the post- instruction interviews. In the pilot study, the explicit and implicit genre groups participated in post-instruction interviews as both groups received instruction in the classroom. As in the main study the explicit genre-group commented on the lessons throughout the instruction, I felt that the post- instruction interviews were no longer necessary. I only analysed the comments the participants had made throughout the lessons (Appendix 5), which I captured through audio-recording and note-taking.

100

The sixth change regards the diary. As was explained above, what I considered a diary in the pilot study was my notes of the points I was unable to capture through audio-recording. For the main study, I decided to drop the term ‘diary’ and simply used the term ‘notes’; the term ‘field notes’ would probably be more appropriate. Having explained the changes that emerged from the pilot study, I now turn to the details of the methods that remained for the main study.