Rationale for the use of CAQDAS software

CAQDAS software allowed us to offer a greater level of objectivity to counter the accusations of bias which are often a criticism of qualitative research. When we coded data, for example, an outside observer may be able to suggest that we chose only data which matched our

interpretations of it. As Fielding ( 2002:172) states ‗Critics of qualitative research cite its lack of formality and cumulativeness…These traits compare unfavourably with the formal and systematic character of statistical analysis and survey methods‘. CAQDAS software allowed us to approach coding in a way which was more systematic and thus more objective than the manual coding used in the pilot study. It enabled us to create categories and move data between these categories easily. This meant we could review our codes many more times than if we had coded manually and attempt to ensure that they were not a partial or biased interpretation. CAQDAS software also allowed us to search for words and phrases most commonly used within a set of data. For instance, if we made a code based on ‗practice‘ we could search the entire data set to find all mentions of this word. This also enabled us to use frequency data to support the ways in which we categorised the data, adding a systematic and objective dimension to our data analysis. These advantages are summarised by Kelle (2002:486):

CAQDAS also helps with the systematic use of the complete evidence available in the data much better than any mechanical system of data organisation. If the data are methodically coded with the help of software, researchers will find evidence and counter-evidence more easily. This clearly reduces the temptation to build far- reaching theoretical assumptions on some quickly and arbitrarily collected quotations from the material.

Naturally, there have been concerns about the use of CAQDAS software as Kelle (2002:478) outlines. Chief amongst these seems to be that the software could somehow remove the researcher from ‗closeness‘ to the data. Kelle (2002:478), however, argues that the technology simply makes clearer some of the problems of qualitative data analysis, chief amongst these being the relationship between the data and theory. CADQAS software did not replace our own data analysis in this study, it simply made coding, retrieving and providing objective

justifications easier for us. We could, for example, find comments in learner diaries or focus groups which seemed to suggest that one group of learners found an activity more useful than the other group. NVIVO helped us to find these comments quickly and easily, and Compleat Lexical Tutor (2011) provided objective support in the form of frequency counts, showing that one group did, for example use the word ‗useful‘ more times than the other group.

4.5 Chapter summary

This chapter has outlined the theoretical tradition upon which the methodology of this study is based. It has also described the research methodology used in the main study and given a rationale for each method used.

The changes made following the pilot study can be summarised as follows:

 It was decided to run a control group alongside the III and PPP groups, something which we were not able to do with the pilot study as there were not enough students available. This group was not given any explicit focus on the DMs but were taking classes at the same time. It was expected that DMs would feature within their classroom and non- classroom input (for example, within teacher talk) but there was no explicit or implicit teaching of them.

 The number of participants was increased to include three groups (PPP/IIII/Control) of twelve learners, (fourteen male and twenty two female).

 Participants were monolingual Chinese learners at B2 (CEFR) level, taking part in a three week pre-sessional academic English course. They had been placed at this level using a standardised placement test (not including a speaking component). Learners had been in the UK for an average of three weeks at the start of the study. The average age of the learners was twenty two.

 A free constructed response speaking test was used as a pre-test, immediate post-test and a delayed post-test, which took place eight weeks after the study. The pilot study did not employ a delayed post-test.

 Two focus groups of six learners from each group were interviewed following the study. Each group consisted of three male and three female learners. Participants were chosen based on availability, ability and willingness to take part. Each focus group was interviewed using a semi-structured interview format, as used in the pilot study.  The diary data and focus group data was analysed using CAQDAS software (NVIVO 8)

to code and retrieve the data. The software made the coding process easier but the actual codes were decided by the researcher‘s interpretation. A corpus programme (Compleat

Lexical Tutor 2011) was used to produce frequency lists, keyword lists, lists of the most frequent lexical chunks and concordance lines of important frequent words.

Aspects of the study that did not change can be summarised as follows:

 Each experimental group received ten hours of instruction and the lessons used were the same as in the target study. An outline can be seen in appendix one.

 The target DMs remained the same, with the exception that ‗well‘ used to close topics or a conversation was not taught.

 The III and PPP frameworks were differentiated in the same way. III involved no practice of the target DMs but did involve tasks which encouraged learners to notice aspects of the language such as the difference between these spoken forms and written forms. The PPP groups were given pre-communicative and communicative practice using the target DMs.

 Participants in the III and PPP groups were asked to keep a diary throughout the course of the study, detailing their views of classroom methods and content. All learners were given a model diary sample to read before completing their own diary entries and were asked to comment upon the class content and methodology. All participants were asked to complete the diaries after each class and they were collected at regular interval throughout the study.

Having discussed and summarised these changes, the methodology used should now be clear. The next chapters therefore display and discuss the results of the main study. We begin by analysing the quantitative data in chapter five before moving on to the qualitative data in chapter six. Chapter seven analyses the results as a whole and the limitations of the study. Finally, chapter eight discusses possible implications for future research and our final conclusions.

5 The main study: quantitative data analysis 5.0 Chapter introduction

This chapter describes and analyses the quantitative data from the main study before the qualitative data is discussed in chapter six. The data analysis in this chapter will include discussion of the results and will relate it to our research questions. The data will also be analysed as a whole in chapter seven, where it is related back to the research questions in more detail and conclusions are drawn. The quantitative data will be presented first through

displaying raw scores from each group. This entails analysing the means and gains made in the interactive ability, discourse management and global scores and the total amounts of the target DMs used by each group. Following this, we will present the one-way ANOVA results which indicated statistical significance. Results which did not indicate significance will be

commented upon but not displayed as tables.

5.1 Interactive ability, discourse management, global marks and discourse marker