Reliability and Validity

Questions regarding criticisms and weaknesses of CDA as an approach have already been discussed in detail in Chapter Three. CDA as an interpretive enterprise leaves ample scope for the validity of the analyst’s claims to be subjected to scrutiny and contestation. Despite differences in the model applied in this study and Gee’s (2011) model for CDA,12 _{Gee advocates four key areas to be scrutinized to enhance the validity}

12 _{Gee’s (2011) model for CDA identifies seven ‘building tasks, that require the analyst to ask the following} set of questions related to each building task but acknowledges that although not all of these may be necessary to raise, a critic can raise the unanswered questions as a criticism.

1. Significance: How are situated meanings, social languages, figured worlds, intertextuality, Discourses, and Conversations being used to build relevance or significance for things and people in context?

2. Practices (Activities): How are situated meanings, social languages, figured worlds, intertextuality, Discourses, and Conversations being used to enact a practice (activity) or practices (activities) in context?

135

of claims made by the researcher. These are: convergence; agreement; coverage; and linguistic details (2011, p. 123). Convergence refers to the compatibility of the findings between each of the areas of textual analysis and contextual analysis identified.

Agreement relates to the level of conviction “native speakers of the social languages and members of the [d]iscourses being implicated in the data agree that the analysis reflects how such languages actually functions in such setting” and the more other data analysts support the conclusions reached also increases the validity. Inevitably this is something that takes time until other research is published on similar data. Thirdly, coverage is concerned with the how much the analysis and findings can be applied to related sorts of data (e.g. other informational / news products). “This includes being able to makes sense of what has come before and after the situation being analyzed and being able to predict the sorts of things that might happen in related sorts of situations.” (Gee, 2011, p. 123). The fourth area of validity refers to linguistic details. The validity of the analysis

increases the more it is tied to the linguistic structure. (Gee, 2011, p. 123). In other words, the analyst must be able to produce linguistic features from the text as evidence for the claims being made. In addition, to the above four criteria, also relevant to the aims of

3. Identities: How are situated meanings, social languages, figured worlds, intertextuality,

Discourses, and Conversations being used to enact and depict identities (socially significant kinds of people)?

4. Relationships: How are situated meanings, social languages, figured worlds, intertextuality, Discourses, and Conversations being used to build and sustain (or change or destroy) social relationships?

5. Politics: How are situated meanings, social languages, figured worlds, intertextuality, Discourses, and Conversations being used to create, distribute, or withhold social goods or to construe particular distributions of social goods as “good” or “acceptable” or not?

6. Connections: How are situated meanings, social languages, figured worlds, intertextuality, Discourses, and Conversations being used to make things and people connected or relevant to each other or irrelevant to or disconnected from each other?

7. Sign Systems and knowledge: How are situated meanings, social languages, figured worlds, intertextuality, Discourses, and Conversations being used to privilege or disprivilege different sign systems (language, social languages, other sorts of symbol systems) and ways of knowing? (2011, pp. 121-122).

136

CDA is the notion of fruitfulness of the analysis.13 _{Jorgensen and Phillips explain that} fruitfulness emphasizes the importance of producing new knowledge “that may foster new types of thinking and action.” (2002, p. 172) but acknowledge that how this is applied can vary depending on the researcher’s perspective on the status of scientific knowledge and her/his perspective on what it should be used for (p. 173).

The issue of reliability is not as great a consideration in qualitative research as it is in quantitative research as there is a “fundamental assumption of multiple, changing realities” which asserts the importance of the researcher’s own continually changing understanding of the object of study (Lindlof & Taylor, 2011, p. 272). Furthermore, CDA acknowledges that each researcher is likely to interpret and read the data (newspapers) differently (as indeed are the wider public), and therefore, CDA recognizes and embraces the variation that can occur in different and competing interpretations. In this way, CDA practitioners differ from the recommendations of Lincoln, Guba and Shenton (cited in Shenton (2004, p. 64)) for ensuring the trustworthiness of research by adopting constructs that correspond to criteria employed in a positivist tradition (e.g. credibility in preference to internal validity; transferability in preference to generalizability; dependability in preference to reliability; and confirmability in preference to objectivity).

Despite the differing views on how to ascertain validity and whether there is a universally acceptable set of criteria to determine it, Jorgensen and Phillips stress, the importance of the researcher explicating the validity criteria s/he intends to follow so that the reader is able to critique and evaluate the findings before accepting or rejecting the claims (2002, pp. 173-147). In addition, the inclusion of supporting evidence in the form

13 _{Jorgensen and Phillips (2002) examine Potter & Wetherell’s (1987) criteria for fruitfulness as potentially} useful for discourse-related qualitative research (pp. 172-4).

137

of excerpts and quotations allows readers to make their own assessments of the analyst’s interpretations as well as allowing them to make their own.

138