Reliability, validity and generalisability

The research constructs of reliability, validity, and generalisability are rarely explicitly discussed within CA studies. This may be due, in part, to CA’s radically diverging position from much research work in the social sciences, and its adherents’ sense of internal cohesion for CA as an enterprise, with the perceived ‘quality’ of research within this approach relying on its ‘comparison’ with other research, which takes the same methodological stance.

However, there is a need within studies, such as this one, to discuss CA as a research methodology and place it within the broad spectrum of other research methodologies and approaches. Much of this section, which attempts to outline these aspects of the

methodology, will draw upon Seedhouse’s (2005b) work, one of only a few papers that directly address these aspects of CA as a research methodology in these terms.

Seedhouse (2005b) suggests that the central issues for CA with regards to reliability lie with regards to the choice of what is recorded, the technical quality of the recordings, and the development of sufficiently detailed transcripts. Another area of importance in considering the reliability of research is whether studies are repeatable or replicable. In CA, unlike most other methodologies, it is standard practice to provide extracts of transcripts of the raw data as part of the analysis. This allows the reader to analyse the data themselves and thus the reliability of CA research is strengthened. Furthermore, it is standard practice within CA to share data with other researchers in private, as well as in public data sessions, such as the ‘Micro-Analytic Research Group’ (MARG) data sessions at Newcastle University, where groups of researchers have, on many occasions throughout the course of this project,

discussed extracts of data, offered critiques on analytic claims and debated several aspects of the analysis within this study.

Seedhouse (2005b) discusses four aspects of validity - internal, external, ecological and construct - in relation to CA research. Internal validity attempts to demonstrate that the explanation or analysis of data “can actually be sustained by the data” (Cohen et al., 2007: 135); it is therefore concerned with the “soundness, integrity and credibility of findings”

(Seedhouse 2005b: 255). There are several aspects of CA analysis that contribute to the internal validity of its research studies, often relating to CA’s insistence on developing an emic perspective (see section 3.3.2). The first of these is CA’s adherence to the fine-grained micro-detail of analysis, demonstrated in the detail of CA transcripts. It is therefore beholden of the analyst to demonstrate that any analytic claims are based on the social actions of the participants and the ways in which they emically orient to these actions. The second aspect is that CA tends to avoid the use of existing theories, from say psychology or linguistics, as these would replace the emic perspective with an analysts’ etic one (see section 3.2.2).

Thirdly, CA does not assume that aspects of context that may seem omnirelevant from an etic perspective, such as gender or power, are relevant to the analysis, unless the participants demonstrate an orientation to these factors in their talk-in-interaction (see section 3.6.1). It is by a keen adherence to these factors that CA work maintains its strong internal validity.

It is also CA’s insistence on the development of an emic analytic position that strengthens its ecological validity in comparison with other research methodologies. Ecological validity is the extent to which the practices of the methodology relate to the reality of people’s everyday lives. This is often weak in social science research, particularly where it treats its participants as experimental subjects. However, the CA practices of recording naturally occurring data and taking a ‘holistic’ participant perspective on the analysis mean that its analyses are firmly grounded in the ecological system being analysed (Seedhouse, 2005b).

Of the other three aspects of validity, external validity is concerned with the extent to which analytic findings can be generalised outside of the specific research context of a particular study. The generalisability of a CA study is dependant on the type of CA research.

Seedhouse (2005b) reminds us that although CA studies analyse data at a micro-level, they are also able to offer certain aspects of more generalisable descriptions of the interactional organisation of a particular setting. This is because the social goals of a particular

interactional context shape the way in which interaction in that context is organised

(Levinson, 1992); the ways in which the context-free ‘machinery’ of interaction is oriented to within a particular context may therefore be generalisable. For example, the close description of the interactional organisation of the feedback meetings in this study, and the explication of their reflexive relationship with the institutional goals of this setting, may potentially be

drawn upon in order to inform practitioners in other contexts where similar feedback meetings occur.

The final aspect of validity, construct validity, is the consideration of whether a research construct is, in actuality, as the researcher claims it is. When applied to CA research, this leads to the question: “who’s construct is it?” (Seedhouse 2005: 257). The nature of CA research with its focus on uncovering the constructs that the participants orient to within interaction, allows it to analyse the social phenomena drawn upon by the interactants and therefore the constructs are ‘real’ and relevant for those interactants, at that given moment in the interaction.

3.7 Summary

This chapter has introduced and outlined the chosen methodology of this study, conversation analysis, as well as providing a brief rationale for this choice. It has described the approach that CA takes to data: its collection, transcription and analysis, and offered a synopsis of the methodology’s roots within ethnomethodology. The impact of ethnomethodological

considerations on this specific study have also been discussed. This led to the description of key interactional organisations within CA and their relevance to the current study. The chapter then moved to introduce the branch of CA which investigates institutional talk-in- interaction. It highlighted the underlying stances of ‘applied’ CA, including its attention to the reflexivity between talk-in-interaction and institutional goals. In doing so, particular aspects of interaction were highlighted as appropriate areas for analytic attention. These areas were related to aspects of the data investigated in this study. This section was closed with a discussion of the implications of applying CA to institutional talk and the ways in which this type of research project can inform practitioners and their practices.

The chapter then turned to aspects of the research design of this particular study. It considered how the methodology of CA is employed in the current study, describing specifics of the research setting, participants, and ethical considerations; as well as a

description of the processes of data collection, transcription, and analysis. The final section of this chapter positioned CA within the broader spectrum of research methodologies. It acknowledged critiques of CA existent in previous literature, considered the issue of

researcher reflexivity and how this relates to the current study, and closed by discussing the areas of reliability, validity, and generalisability. The following analytic chapters are the heart and soul of this study. They will focus on two broad areas of analysis, the overall structural organisation of the feedback meetings and the ‘doing’ of reflective practice as an interactional activity, which in turn relate to the models of reflective practice and the experiential learning cycle, as it is employed to structure the process of RP within this context.