147 Key Readings
*Atkinson, P. (1992) The Ethnographic
Imagination: Textual Constructions of Reality. London: Routledge.
Geertz, C. (1988) Works as Lives: The
Anthropologist as Author. Cambridge:
Polity.
Malinowski, B. (1922) Argonauts of the
Western Pacific. New York: E.P. Dutton.
Malinowski, B. (1967) A Diary in the Strict
Sense of the Term. New York: Harcourt
Brace.
Seale, C. (1999) The Quality of Qualitative
Research. London: Sage.
Reliability/Validity
Definition
Reliability is the extent to which research produces the same results when repli- cated. Validity is the extent to which the research produces an accurate version of the world.
Distinctive Features
Scientific research is typically evaluated using measures of rigour such as reliability, validity and generalizability. However, compared with quantita- tive research, qualitative research has been less concerned with the traditional application of these measures.
Reliability is concerned with the extent to which research findings are reproducible, that is whether a different researcher who replicated the study would come to the same or similar conclusions. It can be argued that reliabil- ity is an impossible criterion to achieve in practice as different researchers will always produce different versions of the social world. Strategies to improve reli- ability include maintaining meticulous records of fieldwork and documenting the process of analysis (in a research diary or in analytic memos) so that others can follow the process in the form of an audit trail. Reliability of the data analysis can be improved through the rigorous comparison of coding of the same data by multiple researchers (Silverman, 1993), thereby resolving ambi- guities in coding by discussion among the researchers.
When considering the validity of the conclusions of a research project, two types of inferences are involved. The first of these is the internal validity of the study. This is the degree to which the investigator’s conclusions correctly portray the data collected. The other inference concerns external validity (also referred to as generalizability). This is the degree to which conclusions are appropri- ate to similar populations and locations outside of the study area.
Strategies to improve validity include triangulation, or member valida- tion, although these strategies are not without their problems. Some researchers, particularly those working from a conversational analysis and postmodernist perspective, append the original data transcript to the research paper in order to allow readers to reach their own interpretation. Anthropologists have used other measures of validity for their findings following lengthy immersion in the field which enhances the accuracy of their account. These strategies include the ability of the anthropologist to ‘pass’ as a collectivity member (Goodenough, 1964) and the prediction of native taxonomy classifications by the anthropologist (Frake, 1961). Validity can also be improved by thorough data analysis in which the researcher searches for deviant cases, thereby revising the theory in the light of the data. Denzin (1989) has argued that from a qualitative perspective, validity reflects a need to provide an improved understanding of the research subject rather than improved accuracy.
An alternative (and positivistic) way of distinguishing between reliability and validity is to think of reliability as a measure of precision (the degree to which a research finding remains the same when data are collected and analysed several times) and to think of validity as a measure of accuracy (the degree to which a research finding reflects reality).
Examples
Barrett and Wellings (2002) (discussed in Green and Thorogood, 2004) provide an example of how the reliability and validity of a qualitative study can be maximized in their study of how women use and define the term ‘unplanned pregnancy’. To improve the validity of their research the authors provide direct quotes along with the context (for instance the point in the interview) to enable the reader to see and judge how interpretations are made from the data, and they report deviant cases to demonstrate how such cases can still be accounted for in their interpretations. For example they discuss in detail the one case in which a woman reported that she had intended to become pregnant although the pregnancy itself was unplanned. The authors do not report the use of member validation in their research. To improve reliability the authors provide a clear account of the data collection and analytic procedures used. They do not report the use of multiple-coders for the same data but they do report dis- cussion among the researchers to resolve the meaning of individual researchers’ interpretations. This study also demonstrates how qualitative research can problematize the validity of survey research. Specifically the authors have con- cerns about how questions in national and international surveys of pregnancy are interpreted by women.
Evaluation
Advocates of qualitative research argue that it cannot and should not be judged by conventional measures of quality such as validity, reliability and generalizability. They reject the position of naïve realism: the belief that there is one unequivocal social reality which is independent of the researcher and can be reached through the rigorous application of the scientific method. Instead a relativist position is adopted reflecting the belief that there are mul- tiple perspectives of the social world and these are constructed by the research process. Middle ground has been found between the extremes of realism and relativism. These intermediate positions are referred to as ‘subtle realism’ (Hammersley, 1992) and ‘critical realism’ (Bhaskar, 1989), amongst other terms. These positions reflect an acceptance that although the social world is perceived from a particular viewpoint, some of these viewpoints are more plau- sible than others. Consequently qualitative research will still need measures of quality on which to judge plausibility. Lincoln and Guba (1985) have argued that for qualitative research the positivist concepts of validity and reliability may be replaced by criteria of truth value, applicability, consistency and neutrality. But as Seale (1999) points out, terms such as ‘truth’ do not fit well with the position of relativism which supports the idea of multiple constructed realities rather than the realist view of a single tangible reality. Guba and Reliability/Validity
Lincoln have acknowledged this problem in later writings (Guba and Lincoln, 1989) by providing another criterion, ‘authenticity’. Alternative evaluation measures have also been suggested by Hammersley (1990).