Interview and corpus working session data: Thematic analysis

In the analysis of interview information, I used a constructivist approach in which “interviewers and interviewees are always actively engaged in constructing meaning”

(Silverman, 2006, p. 118) because an underlying principle of this research is that the knowledge we seek exists as part of the teacher’s set of beliefs and knowledge base (Borg, 2003); hence, approaching the interview as an opportunity to understand how these teachers view the world as language teaching professionals could provide them the room to explore what they may not have even acknowledged previously.

In using thematic analysis through the use of coding sections of texts, it is important to acknowledge that “coding necessarily reflects the researcher’s interests and perspectives as well as the information in the data” (Charmaz, 2002). In other words, it would be disingenuous at best to make any claim that I enter this process unbiased or having somehow excised my prior

experience and knowledge. While in quantitative analysis this acknowledgement may be considered a liability, in this kind of grounded qualitative research, acknowledging and even embracing my role in the process can make the analysis more powerful, with “the personal-self becom[ing] inseparable from the researcher-self” (Creswell, 2003, p. 182) . As Charmaz (2002) suggests, as a researcher I can use my own pre-conceived ideas about teaching and the use of corpus tools as an effective starting point for further refinement, provided I do it reflectively and with self-awareness. This is in stark contrast to objectivist researchers who “minimize…

reflexivity about the constructions- including preconceptions and assumptions- that inform their inquiry” (Charmaz, 2002, p. 683).

The first step in working with the interview and corpus working session data was to transcribe each interview and corpus working session, making notes of distinctive elements or anything that might be unclear to someone reading the transcripts later. For example, many of the codes for corpus tools deal with specific online tools, such as the Corpus of Contemporary American English (Davies, n. d.), which might not be clearly understood by someone who is not well-versed in that area. In such instances, I added parenthetical information that provided the necessary background knowledge. As Braun and Clarke (2006) noted, “what is important is that the transcript retains the information you need, from the verbal account, and in a way which is ‘true’ to its original nature” (p.88). As another example, during Lorraine’s first interview, she began to make reference to the textbook on her desk. This exchange was unclear in the

transcripts, until I added additional information to note that she was referring to a specific book; it also occurred when she began discussing her syllabus and course materials. I reviewed all transcripts for accuracy by reading them while listening to the recordings again. Listening to the recordings several times, as suggested by Braun and Clarke (2006), allowed me to become even more familiar with the way in which each teacher described her own process of approaching the course, and the reasoning behind it.

Member checking, or providing participants with data at various points in the research process for their review and feedback, is an important element of building confidence in a researcher’s analysis of qualitative data (Ryan & Bernard, 2003). After my review and revision of the transcripts, I sent them to the participants, as a first step in the process of member

checking (this topic is further addressed in Section 3.3.2 below). My goal in sending the transcripts was to provide each participant with an opportunity to identify any sections that she felt misrepresented her intent in the interview. In fact, all three teachers found that the interview and corpus working session materials accurately represented their voices and their views.

After transcribing and verifying the data from the interviews and corpus working sessions, the primary tool for analysis used was thematic coding, “a form of shorthand that distills events and meanings without losing their essential properties” (Charmaz, 2002, p. 684). I examined the data using Atlas.ti thematic analysis software (GmbH, 2013) and began an initial coding stage. The steps I used in the coding process were to first do initial coding without referring to the literature and without setting any limits on the scope of the possible codes. As suggested by Charmaz (2002) this coding was done line-by-line, with some codes being

generated through keywords in the text and others being generated by the underlying meaning of phrases, sentences, or larger sections of discourse.

This coding process was then repeated, as recommended by Charmaz (2002), so that new codes could be added and initial codes could be rejected as inaccurate, too vague, too specific, or unimportant within the larger context. In addition, the codes could then be grouped and

regrouped, building sets of larger codes or themes based on categories from teacher cognition, such as teacher beliefs, decision-making, or assumptions about learners, or based on categories from the use of corpus tools in teaching and learning, which could include limits on corpus tools, ease of use, or ideal corpus tools.

This initial coding resulted in numerous comments on the ideas that the participant expressed. For example, Figure 4 shows how Cecilia responded to the question “How do you think the lesson went?” during the third interview.

1 2 3 4 5 6 7 8

I think it went pretty well. I would have felt better if I'd had more of a resolution task for them, other than the vocabulary task, because when I got that back, and only four or five people did, I got back words they should have known already or I know that they know already. So I think if I do it again I want another final task that gives it more closure, or maybe even asks them their opinion about it. Or require them to give their opinion, requires them to freewrite about it- I don't know what it would be. But I feel like it wasn't a total void of resolution, but I felt it was a little bit unresolved. I think they were game, I think they did

really well with the overwhelming visual nature of what we showed them. (C3-0151)

Figure 4 - Excerpt from interview data

During the transcribing process, I added a note “not really sure she will do this again!” at Line 4, based on her tone of voice when she said “So I think if I do it again...” At Line 5, I noted “her focus on student affective issues or student perspective on the

relevance/effectiveness/success of activity.” Then, in the initial post-transcription analysis, I added comments to various sections of the response. For Lines 1-3, I added the note “wants better ‘ending’ task.” In Lines 4-5, I added the note “wants student view” and in lines 5-6 “make it a writing task.” For lines 7-8, I added the note “critique of website.” In further review of the data, I began to add more nuanced codes. It also was becoming apparent that much of the thematic content went longer than one or two lines, so for this section, the entire passage was coded as addressing “reflective practices,” “student affect,” and “teacher reasoning.” The final

1 _{Excerpts are identified by the teacher initial, interview session, and paragraph number. This}

sentence (Lines 7-8) was coded for “Lextutor” (the corpus tool used), “comfort-students,” “distraction,” and “student affect.”

During the reiterative process of coding, a final set of 111 codes emerged. These codes were then gathered into eight larger categories, or code families: corpus tools; lexicogrammar; material design, selection, and critique; student issues; teacher cognition; teacher issues; technology; and writing. The number of codes within each family varied, ranging from the technology family having only one code (technology), to the teacher cognition family having 29 codes. The categories and codes together with working definitions and illustrative examples from the transcripts are laid out in Table 5.

Table 5 - Categories and codes in thematic analysis Category and codes Working definition Examples Corpus tools Codes: concordancing, corpus 'dream tool', corpus creation, corpus effectiveness, corpus limits, corpus linguistics, corpus research, corpus tools, DDL, keywords, (and various specific corpus tools) Statements that address opinions, knowledge, or use of corpus tools by the teacher

Code: corpus limits

I don't know if I really would [use the corpus tool again]. If there were other options, I might not. I think it's not the most user-friendly website going. As a website, as a visual tool, it's overwhelming. I have an issue with some of the statistics, also. I'm not so sure about the value of some of the

information that you get, and I'd have to - some of the feedback that you get- I'd have to work with it more.

(C4-20; additional codes: material critique, teacher reasoning) Lexicogrammar Codes: conditionals, grammar, pronouns, AWL, phrases, pronouns, reporting verbs, summary words, transition words, vocabulary, word forms, word lists

Statements that address grammar or vocabulary issues

Code: vocabulary

I think that it's hard for them to otherwise be aware [of new vocabulary]. They only know the words they know; they don't know the words they don't know. So how else would they find out the words they don't know? I wish there were some similar mechanism to help them learn word forms. I think learning morphology, learning over the long term, helps them, but I think they really have trouble at this level with accurate word forms. And it really often obscures the meaning in their writing. (C3-054; additional codes: corpus effectiveness) Material design, selection, or critique Codes: material critique, material design/selection Statements that address how the teachers view materials (both actual and potential)

Code: material critique

I take back what I just said, because I think it depends on the student audience. I do feel, for the [graduate student] audience, I just feel as if that could be useful. And the way that [the textbook authors] have created other books, there's a lot in there that you're not going to use at all. So I think incorporating corpus tools into a book like that- I think could be helpful. But I don't know.

(L4-084; additional codes: corpus effectiveness, L2 writing, student-as-consumer model, student level, teacher reasoning)

Student issues Codes: comfort-students, curricular issues, distraction, feedback to students, fun, impact on students, student-as- consumer model, student affect, student autonomy, student buy-in, student differences, student goals, student interaction, student level, student needs, student strengths, students-as- linguists concern Statements that address student needs, concerns, and interaction

Code: student level

Next fall, I think I'll teach an undergrad class. I'm going to try [using the corpus tool], but I think I will use it more in the graduate class. I think that audience tends to be more motivated. They have a real need to be better writers. It's very apparent to them that they need to be able to write effectively, and they want to write like others in their field. So I think that this is more interesting, perhaps, to grad than undergrad. I say that based on- I've never tried it with undergrads, but I've heard other people who have, and I was told that the undergrads weren't all that interested in it. But I'll try it.

(L4-007; additional codes: integration into

teacher’s system, L2 writing, student affect, student buy-in, student differences, student goals, teacher reasoning)

Teacher cognition Codes: ambiguity, beliefs, buzzword, collaboration, concerns, cost- benefit ratio, dislikes, future goals, goals, innovation, integration into teacher's system, knowledge base, likes, needs, past beliefs, patience, pedagogical knowledge, personal experience, personality, present beliefs, professional development, reflective practices, self- awareness, self- doubt, strength, teacher 'face' with students, teacher affect, teacher reasoning, weakness Statements that address teachers’ cognitions Code: beliefs

I believe that you have to be flexible. I often will talk about what works for me, with my students, as a writer, but then I want to hear from them because I believe there's not just one approach to something. And I think that may be an area where I've grown a bit. Maybe I try to be less, well, rigid I think is more negative- but I try to be more- I keep saying flexible, but realize that we can approach things in more than one way…. So I guess that would be a belief- that you have to change… My belief is that, as the instructor, you have a plan but the student is the one that you're teaching to, and so you might need to- you have to listen to what they're saying and you have to seek feedback from them and be willing to alter what your plan was.

(L1-25; additional codes: classroom management, flexibility, impact on students, student differences)

Teacher issues Codes: administration, assessment, classroom interaction, classroom management, collaboration, concerns, control, curricular issues, feedback to students, flexibility, grading, language issues, learning outcomes, lesson plan, office space, practicality, program concerns, resources, time, workload concern Statements that address the teacher’s role, actions, and issues in handling students, resources, and the institutional environment Code: time

We’re now working on problem-solution writing, and [the students are] doing their proposals, but I don’t know, because it’s not so smooth. This took too much time because I was trying to figure out what was wrong, and I said “let’s stop doing this, and you’re going to…” I think if I tried this, I could maybe condense it by one week. I feel like I wasted two class periods, not back to back but a little this in one class... over time I wasted two class periods because I was trying to figure out where’s the disconnect. What’s happening here? Why are there so many questions?

(L2-24; additional codes: academic writing genres, L2 writing, reflective practices, self-doubt)

Category: Technology Codes: technology Statements that address knowledge or use of technology by the teacher or students Code: technology

[Technology has] become very important but there is also the hindrance [of] using technology, like for example the computer lab and the student's desire to be engaged with their email or Facebook, but I think it's been real helpful having students be able to word process. If you think about years ago, typing, and now that they can just edit and change within their [document] and also they are able to use the internet to find sources and support research for what they're writing about... and then the corpus stuff, and just the way you are able to communicate with students too, through email or using the [online course management] support system. (S1-43; additional codes: corpus tools, distraction, student affect)

Writing Codes: academic writing genres, attribution, citations, coherence, cohesion, content knowledge, elements of good writing, genres, hedging, L2 writing, models of writing, organization- writing, paraphrasing, plagiarism, process writing, timed writing Statements that address issues of academic writing

Code: models of writing

I also [like] looking at good models of writing, especially student models, so I really like when I've got examples of good writing that they can see other students like them wrote. So I think looking at models and writing... and trying to simulate- and a lot of the writing we ask them to do, it's not creative writing, you know- it's following a formula. And you can be creative within that, so I think that's the challenge too, that they can see, OK, here's the formula, and if they want to be creative within that, they can, you know in terms of adding material. (S1-34; additional codes: L2 writing, student needs)

From the analysis of these codes, themes began to emerge. The most pressing needs for the three participants collectively were time, student issues, material analysis, selection, and design, issues with the corpus tools, institutional issues, and collaboration. These themes are discussed in Section 8.1 of Chapter 8.

An important component in qualitative teacher education research (see, for example, A. Baker, 2011; J. Lee, 2011) is to implement a process of inter-coder agreement (Willis, 2007), or what Lee (2011) called his “external check” for thematic coding (p. 79). One way to build consumer confidence in thematic data analysis is to demonstrate second-coder reliability, or “the degree to which coders agree with each other about how themes are to be applied to qualitative data… [which] suggests that the concept is not just a figment of the investigator’s imagination and adds to the likelihood that a theme is also valid” (Ryan & Bernard, 2003, p. 104).

To work towards this end of building consumer confidence through a second coder, I recruited another researcher, a doctoral candidate who was also working on a qualitative study of

language teachers. We held a training session in which I explained my research questions and how I had started coding the initial sets of interview data. We then went through several excerpts of interview material together, discussing the coding process and clarifying concepts as needed. In this preliminary stage, we examined interview and corpus working session data using excerpts from all three teachers and from all stages of the research: these stages included the initial

interview, the second interview focused on the specific class and students, the corpus working session, the immediate follow-up interview with the stimulated recall component, and the final interview after several months had elapsed.

Once the second coder understood the method and scope of coding and we went through some coding together, I provided her with a sample of 5,573 words, or about 10%, of the

interview material to review on her own. Over the period of several weeks, she then coded the material using my list of codes as well as other codes that she might create, and we met again to compare results. Having inter-coder agreement of 90% is considered optimal (Miles &

Huberman, 1994) but on our first review, our agreement rate was at about 76%. We further discussed the coding topics related to corpus linguistics, as she was not well-versed in corpus tools or terminology, and we realized that the main disagreements in coding were related to the corpus tools. The codes that I had created based on the specific content of the study were: Corpus of Contemporary American English (COCA), concordancing, corpus 'dream tool', corpus

creation, corpus effectiveness, corpus limits, corpus linguistics, corpus research, corpus tools, Data-Driven Learning (DDL), keywords, Lexchecker, Lextutor, and Michigan Corpus of Upper- Level Student Papers (MICUSP). We discussed these codes in more detail, and I explained their role in the study more fully. Once she became more familiar with those codes, and based on our