Data collection and analysis methods

The following section offers a brief description of the methods of data collection and analysis used in this research. Data is collected using surveys, interviews, field notes and audio recordings. The classroom audio recordings will be analysed using conversation analysis.

4.10.1 Surveys

Students were surveyed at the beginning of the first iteration to gauge the feasibility of the Talk Skills intervention with regard to students’ need for improved discussion skills and their attitude towards different elements of the Talk Skills intervention, for example, making ground rules or being taught strategies. The survey was translated into Korean and piloted before being administered.

Dörnyei (2003) points out the need to be aware of the potential limitations of survey use for data collection. Of those relevant to this research, firstly, while some respondents may put time and effort into their answers, others may be unmotivated to do so, as the process offers no actual benefit. Secondly, while the survey in this research was offered in Korean, the students L1, some respondents may still find the survey questions difficult to understand. Thirdly, the problem of self-deception (Hopkins et al. 1990) may also be relevant, in that students may, for example, deceive themselves into the belief that their L2 is better or worse than it is, meaning answers may not represent truth. Finally, in line with the acquiescence bias (Robinson et al., 1991), students may also simply agree with whatever they perceive sounds best. Providing the survey is well constructed, it will provide a generally reliable and valid source of data (Dörnyei 2003). Nevertheless, an awareness of the potential problems, and where possible, actions to guard against them, such as encouraging students to answer honestly, making clear the

107

anonymity of the surveys and making clear that responses would not affect any current learning, will help to guard against data misrepresentation.

4.10.2 Interviews

Following recommendations in Bakker and van Eerde (2015), two types of interview were administered during the iterations. The first was a relatively unstructured interview with one or two students at the end of each session of the intervention. These interviews were conducted by myself, in English, to gauge the students’ thoughts about a given session.

The second type of interview took the shape of semi structured interviews conducted at the end of the intervention, to attain an overall view of the intervention from two students each iteration. Each student was offered the use of a bilingual interpreter for the interview, but declined, stating that they were comfortable listening and responding to my questions in English. All interviews were audio recorded using MP3, or smartphone recording devices, then transcribed using Microsoft Word and Windows Media Player.

4.10.3 Field notes

During the designing phase of the intervention and throughout each iteration, I kept my own field notes, which were either audio recorded after sessions of the intervention, or written into a notebook. The field notes were logged systematically (Dörnyei 2007) at the end of each session and covered my thoughts about implementation of the intervention, how I felt the activities were received by the students and potential changes that would benefit the intervention, among other notes. All audio files of recorded field notes were transcribed using Microsoft Word and Windows Media Player.

4.10.4 Classroom recordings and transcriptions

Each session of the intervention was recorded using MP3 recording devices. An MP3 recording device was placed among all pairs or groups of students during each session of the intervention. All classroom interaction of each session was recorded.

108

Later, audio files were carefully played back and one or two examples of activities, plenaries and other instructional elements of each session were chosen for transcription. When choosing transcripts that would be presented in the thesis, best effort was made to include data that was representational of what generally went on among all groups within the class. Data was transcribed using Microsoft Word and Windows Media Player, using transcription conventions outlined in Ten Have (2007), see Appendix V. Transcriptions of classroom interaction were analysed using conversation analysis (CA) methods. CA as a method for analysing EFL classroom transcription data is outlined in the following section.

4.10.5 Conversation Analysis for analysing EFL classroom discourse

A major advantage of CA is that it is able to offer an emic, data driven perspective on the social, interactional nature of language (Sert & Seedhouse 2011). As such, CA aims to “discover how participants understand and respond to one another in their turns at talk, with a central focus on how sequences of action are generated” (Hutchby & Wooffitt 1998: 94). Taking a CA approach to interaction allows for analysis on a turn by turn basis. This process has shown that turns at talk may operate on a number of levels:

“The utterance is a display of the learner’s analysis of the prior utterance of an interactant; it performs a social action in response and it positions the learner in a social system. It displays an understanding of the current context (sequential, social and L2 classroom context) and also renews it.” (Seedhouse 2005: 178) As such, Seedhouse points out that turns within the discourse are complex and are used to talk the classroom context in and out of being. In other words, it is through the interaction that context is both shaped and renewed. Students also use their turns to show their position with regard to understanding the context in which they are in. It is essential to consider the sequential environment in which contributions to talk take place and in which participants are interacting, as it forms the interactional environment and therefore, all details of the interaction should be viewed as potentially important. With

109

regard to the current research, this means that CA can be used as a means to analyse the turn by turn interaction that takes place during the intervention’s activities.

The incorporation of CA in SLA studies, though, has not been without criticism. Criticisms include “CA is a behavioral discipline while SLA studies is a cognitive discipline” (Markee 2000: 30), the counter argument being that cognition may be viewed as socially distributed and observable in conversation, and therefore analysable through a CA perspective. One school of thought suggests that SLA is, at least partly, introspective, passive and singular. In this respect CA is not useful, as it cannot analyse what is not observable. He (2004: 573), however, argues that CA does become useful when SLA is considered not as passive and static, rather as “an active process of problem solving”, as is the case with much classroom interaction. This claim is advanced by Schegloff (1991) who argues that sequencing, turn taking and repair may be seen as socially distributed cognition.

A further criticism suggests that CA may be equipped to examine language use, but not language acquisition (Markee 2000). The negation of this claim, while accepting that language use is subsumed by acquisition, asserts that both are inextricably linked and that SLA studies would, in fact, be enriched by “conversational analysis of the sequential and other resources that speakers use to modify each other’s talk and thereby to comprehend and learn new language” (ibid: 32). The enrichment CA offers is in helping us to understand how the language is learned as it is being used.

The view of CA’s contribution to SLA taken in this current research is in line with Markee (2000: 44), who states that “CA can help refine insights into how the structure of conversation can be used by learners as a means of getting comprehended input and producing comprehended output.” Furthermore, the language learning classroom with language as both the means and the goal of the class, coupled with learners who are not fully proficient in the language, make language classroom participants “display of and orientation towards understanding... critical to the overall purpose and outcome of the talk itself” (Huth 2011: 300).

CA, then, may better our understanding of SLA, in as much as analysis is able to take on an emic perspective of participants’ interactional practices, describe them using fine grained transcripts, use such transcripts to identify evidence of learning and

110

understanding as they occur in conversational behavior and in doing so, add to our understanding of the social interaction hypothesis (Markee 2000).

This current research aims to use applied CA (Kasper & Wagner 2014) as a means of understanding how students interpret the activities within the talk skills intervention and illustrate the extent to which the hypothetical, planned learning trajectory of each of the activities met their respective actual learning trajectories when the activities were carried out by the students. Using CA in this research then, offers an attempt to gain insight into interaction within the given activities of the intervention, by attempting to show whether such interaction, among the members of the classroom is “‘doing’ what we expect [it] to, and how?” (Huth 2011: 300).