Language proficiency and recall

The following extract is from an initial interview with Katy, (BS = the researcher), one of the three student participants in Archaeology. In the later part of this baseline interview, I was gathering data on participants’ knowledge of assignment writing criteria set by the department:

Extract 1

BS: What did Tutor B say to you about the marking criteria…do you remember, anything specific?

Katy: erm… (pause)

BS: …that’s the marking criteria in… she looked at in class with you? Katy: Yeah like, be neutral, or be …writing critical, critical writing…. BS: Mmm

Katy: (laughs)…like that.

BS: What do you understand by critical writing? Katy: Yeah…I try to but sometimes it’s hard…

BS: When, when she said critical writing like that, what did you understand? For you what is critical writing then?

Katy: …(pause) so, give some evidence…um (pause) erm well organised (laughs)..

BS: OK….yeah…well (Katy I:1)

The first issue this extract highlights is that of recall. It is possible that the student has trouble remembering exactly what was said about criteria in the class referred to, and it may not be surprising that she struggles to answer this question. As the

interviews deal with earlier events such as receiving feedback or the experience of writing that took place days and possibly several weeks before the interview, then there is always an issue with the accuracy and reliability of participants’ recall. One

123

way I attempted to counter this was to ask participants to have copies of assignments and feedback with them at interviews. Participants generally complied with this request, but in line with ethical principles discussed above, I did not feel I could insist on this if participants did not bring their feedback to the interviews.

Although I attempted to minimise questions that relied heavily on memory, interviews took place several days after feedback events, so to some extent the issue of recall could not be completely eliminated. A basic approach to controlling for the recall variable was to try to ensure that all students were interviewed at a similarly appropriate point after receiving feedback, within a one-week period and with a delay of no more than one week after receipt of feedback. My strategy was to provide sufficient time for students to digest their feedback, while avoiding at the same time a delay which might affect recall.

Katy’s language level is clearly a constraint in the example above. Her misunderstanding of the first question which asks about the meaning of ‘critical’ seems to suggest a problem with language. Katy’s inability to express herself at length and with fluency in this extract is telling. The need to phrase questions carefully in terms of vocabulary is also clearly highlighted here, though this may vary from participant to participant. The attempt to clarify this point with Katy in this example leads to several questions, and more interviewer talk, while the resulting response is a short broken sentence. In this case, language issues can be seen to affect the quality and richness of the data that the interview can yield.

The example above also shows up the problem of assuming the participant has the required knowledge to answer specific questions, as the direct question on critical writing highlights. Part of Katy’s problem in finding an answer may have been as much due to a lack of knowledge as due to difficulties in expressing herself in English. Indeed, the tacit knowledge issue referred to in Chapter 2 may be evident here, and if tutors themselves find it difficult to articulate what ‘critical writing’ means (Chanock, 2000), then it is not unreasonable for a student in the early stages of her Master’s study to struggle to do so.

124

The second extract below is from a pilot interview with a student on a language education Master’s course carried out in summer 2008 (see Section 4.1) prior to embarking on the preliminary study proper. The student, referred to here as Cindy, was about to complete her programme having already submitted her

dissertation. A question posed later in the interview asked if Cindy could remember any pieces of feedback on her assignments.

Extract 2

BS: Can you remember any feedback, because you were talking about this before…I’m thinking specifically about feedback that you were given on assignments that helped you to develop and improve your writing skills? C: I remember…. The first one is not very easy to read, because it was

unclear… so my supervisor suggest me to have the main sentence for each paragraph, yeah.

BS: So a kind of thesis statement really? C: Yeah... thesis statement

BS: And, you thought that was useful? C: Yeah.

BS: Except you were telling me it didn’t quite work…

C: In the beginning, I didn’t really get it and then in the term two I think it works better…..And the second one is too much information redundant and I repeat a lot…

BS: Any other things you remember from feedback?

C: The third one…because I conducted a study and it’s my first time to do a study so I don’t really good at data analysis I didn’t really do that very well. For me now that’s really helpful because that assignment told me how to do my further study

(Student pilot interview 1)

This extract demonstrates the way a fairly scripted question elicits information that is elaborated on as the interviewer uses probes to follow up the initial response. What is significant here is that the interviewee is able to remember three separate points from feedback that she used to improve her writing. She is also able to reflect on how the development from feedback was not immediate, and how the feedback on paragraph writing only became clearer for her later in the second term. The student is able to express herself more fluently, but this may be related to the fact that the interview took place towards the end of the student’s one year in the UK, unlike the example with Katy, which took place within a few weeks of the start of her

programme.

The implication from the extracts above is that participants may vary quite widely in terms of language fluency and expression, and also in their ability to recall feedback events. This in turn implies that the interviewer’s behaviour in terms of

125

providing prompts and explanation may also vary as a result. Where language is not an obvious barrier to understanding and expression, richer data may be expected, and vice versa. The challenge in design terms was to prepare interview schedules that would avoid over complex language.