Quality within qualitative inquiry

Qualitative inquiry is anything but a soft option—it demands rigour, precision, systematicity and careful attention to detail (Richards, 2003, p. 6).

Methodology, and the methods that arise from it, are social practice, enjoying “no immunity from or

transcendence of the contextual realities governing any activity” (Canagarajah, 1999, p. 54). Reflective research using a given methodology should by definition have something to say about the methodology

85

itself, and the researcher must account for “the ideology which is embedded in its own discourse,

method and theories” (Holliday, 2007, p. 14). This accounting process is a part of quality control, and it

is this issue to which I now turn my attention. In this section I address the qualitative counterparts to measures of quality commonly used within quantitative research: generalisability, validity, and reliability. Opinions from within the qualitative realm vary from those who view such measures as either entirely unsuited to qualitative inquiry (Lincoln & Guba, 1985; Wolcott, 1990) to those who feel they can be appropriated (Hammersley, 1990; Morse, 1999). The only agreement between the camps might be that

debates on quality within qualitative inquiry “find their most vehement, and often most intractable,

expression in the field of criteriology” (Richards, 2003, p. 292). I finish by introducing what I argue are criteria more suitable to evaluate the quality of the current study: resonance, trustworthiness, lucidity, and originality (Charmaz, 2005).

Generalisability

Given that the basic purpose of research is to find the patterns and rules behind the complicated surface, a key concern for qualitative researchers is the extent to which their research can expect to be of relevance to other settings. Richards (2003) writes that for research to be worth its salt it must certainly have relevance to those outside the immediate research settings who are in analogous or related circumstances. The question is whether or not to evaluate this relevance in terms of

‘generalisability’. Lincoln and Guba (1985) consider the term to be tainted by association with its use in quantitative work as a statistical measure; they prefer the term ‘transferability’. Others suggest that

generalisability is an acceptable concept in principle, but needs to take on a different nuance to that which it has acquired within quantitative work. A third option is to highlight the relevance of the particular and to leave the question of whether the results can be generalised to the judgement of the reader. Richards (2003) writes:

86

The success of such research will certainly depend on the extent to which it allows readers to

engage with the situation described and interpretations offered in terms of their own

contextualised experience and its power is more likely to be transformative than merely persuasive (p. 289).

My position is that there is no doubt that a well-done qualitative study can eventually be shown to be relevant to other situations and contexts. However, it is not the case that the criteria against which this relevance is to be judged can be set out beforehand. This position can perhaps be explained through the analogy of a mechanic who takes apart a car and puts it together again. In doing so he can expect to learn something valuable about cars in general. In a similar way, the theory presented here is based on the examination of individuals who, while unique, are not ‘uniquely unique’—they share many

attributes with learners in other contexts, and the theory, if well constructed, is bound to be generalisable to other situations to a degree. The fact that this generalisability does not assume a numerical value is irrelevant. Having stated this, I need to be cautious not to make any unjustified claims for the universality of the ELMS models. The basic model is, strictly speaking, a theory of the five research participants alone, while the generalisability of the extended and LMS models is a ‘hypothesized generalisability’ based on the data, on my own experiences, and my knowledge of the literature. With qualitative work the generalisability of research can only truly be confirmed if and when it has been read, and found to have been insightful and relevant, by researchers working with other students in other contexts. It is the potential reader who will be in a position to make any stronger claims for the models’ generalisability. Nonetheless, the researcher can do all he/she can to maximise the probability the research will be useful and insightful to others. I therefore need to ask myself whether I think my research is likely to resonate with such readers, bringing to mind parallel or related phenomena from their own professional experience, and whether it offers explanations that the

87

potential reader will likely find useful in applying to his/her own context. Rather than generalisability, the criterion I have chosen to ensure quality is Charmaz’s (2005) category resonance.

Validity

In quantitative research validity refers to the extent to which a study measures what it claims to study. Put in such terms it seems reasonable to expect it to apply to qualitative research too. Maxwell (1992) suggests that descriptive, interpretive, and theoretical validity are categories of validity to which a qualitative approach might adhere. Descriptive validity describes the factual accuracy of the account insofar as this is verifiable, interpretive validity describes the degree to which this account reflects the insider/

participant perspective, and theoretical validity describes how well the interpretation functions as an account of a particular phenomenon. Together, these elements provide a framework within which claims in qualitative inquiry can be developed in a principled manner (Richards, 2003). Charmaz’s

(2005) criterion trustworthiness can be thought of as subsuming Maxwell’s categories.

Reliability

Reliability in the quantitative realm is, among other things, a matter of other researchers being able to do the same experiment and producing comparable results. Research based on unstructured interviews is clearly about as far from a replicable laboratory experiment as it is possible to get: such interviews are portraits of particular individuals at a particular point in their lives, and they are the outcome of

interaction between these distinct individuals and the individual researcher. A sensitive interviewer will have taken measures to conduct the interviews in a reasonably objective manner, will try to listen

without judging, and will refrain from interjecting his/her own opinions and narratives. But he/she is still has particular, personal theoretical interests in mind during the interview. The research presented here is very much the product of mine and the participants’ own experiences, including that of the

88

interview itself, and is therefore not in any specific sense replicable. This is not necessarily a problem, however, in the same way that reliability is not a relevant measure of the worth of a good novel or critical essay, for example—reliability is simply one criterion of certain types of scientific research.

For the above reasons, of the three quantitative benchmarks, reliability is the least relevant to a qualitative approach. Interestingly, recent research suggests that it may be a highly problematic concept even in experimental studies. In a widely publicised report, Bohannon (2015) found that, of 100

prominent psychology papers, only 39% could be replicated unambiguously. Charmaz’s (2005) criterion

the nature of the writing is a more relevant criterion than reliability to the type of analysis presented here. Ideally, the reader needs to be aware of exactly what the researcher did, for what reasons, based on what presumptions, all of this being conveyed in unambiguous, concise prose. The reader can scrutinise the research to see if it is useful in some sense to other researchers, or if it is an exercise of ego,

something that says more about the researcher than the object of study. As Seale (1999) writes:

Methodological awareness involves a commitment to showing as much as possible…the procedures and evidence that have led to particular conclusions, always open to the possibility that conclusions may need to be revised in the light of new evidence (p. x).

Naturally there may be a trade-off between showing as much as possible, and other aspects of writing such as readability and restrictions on space.

Resonance, trustworthiness, lucidity, and originality

I conclude this section by providing more details of Charmaz’s (2005) criteria that I referred to above:

trustworthiness, originality, resonance, usefulness, and the nature of the writing. Resonance refers to whether the richness and completeness of the research is portrayed; whether everyday or implicit understandings

89

have been illuminated; whether the minutiae have been linked to the ‘big picture’, and whether the interpretations make sense and contribute to participant understandings. Trustworthiness is largely

synonymous with Maxwell’s understanding of validity. It can also be seen to be of direct relevance to

an exploration of social meanings rather than facts. As Mishler (1990) suggests:

Focusing on trustworthiness rather than truth displaces validation from its traditional location in a presumably objective, non-reactive, and neutral reality, and moves it to the social world—a world constructed in and through our discourse and actions, through praxis (p. 240).

The nature of the writing(I prefer the term ‘lucidity’) refers to the need for lucidity and transparency. I see trustworthiness, resonance, and lucidity as qualitative counterparts to the quantitative criteria validity, generalisability, and reliability. Quantitative work focused on verification does not necessarily have to be original to be good. The same is not true of qualitative work because it has as its focus a particular group of participants or a unique target of study. Originality demands that categories are both data- derived and insightful; that the analysis moves beyond description to a conceptual interpretation; that the researcher shows the social and theoretical significance of the inquiry, and how the work makes a contribution to existing theory.