Framework for ensuring trustworthiness

Shenton (2004, p. 64) proposes four criteria, drawn from Lincoln and Guba’s (2003)

framework, which can be used to ensure trustworthiness in qualitative studies (in preference to notions of validity and reliability favoured by positivist researchers):

a) credibility (in preference to internal validity);

b) transferability (in preference to external validity/generalizability); c) dependability (in preference to reliability);

d) confirmability (in preference to objectivity).

Shenton specifies provisions that can be made by researchers to promote credibility, i.e. confidence that the phenomena being studied have been accurately represented, which can be subdivided into the five broad groups outlined below.

Credibility of the research processes: This can be established through deriving appropriate research methods from those that have been used successfully in comparable projects in the past. Criteria for credibility, established by action researchers, include the extent to which the research processes are participatory, collaborative, relevant, and most importantly, result in positive social change (Brydon-Miller, et al., 2003).

Credibility of the researcher: This can be established by making explicit the background and previous experiences of the researcher and the development of the researcher’s thinking during the project. Having accepted that knowing is from a perspective, ‘critical subjectivity’ involves developing awareness of, and articulating, that perspective (Reason, 1994). From my acceptance of the partiality of my position as researcher, ‘reflexivity’, i.e. “the process of reflecting critically on the self as researcher” (Lincoln & Guba, 2003, p. 283), and critical subjectivity are essential in establishing credibility. Trustworthiness is dependent upon accounting for ‘located perspective’, i.e. “the extent that we can simultaneously consider our subjectivity from a ‘distance’” (Ladkin, 2005, p. 123). Failing to do so can result in self-

deception or reluctance to consider experiences which challenge initial perspectives (Reason, 1994). Maintaining a reflective journal can provide transparency by enabling the researcher to make clear how experiences, values and developing perspectives influence the research process (Ortlipp, 2008). Perspectives can also be made transparent through

“autoethnobiography”, i.e. telling the story of the construction of our own identities and ideologies relating to the research, which rests on the assumption that “the truth of the self is integral to the truth of the study” (Humphrey, 2007, p. 22).

Credibility of the relationships between researcher and participants: This depends upon building the familiarity of the researcher with the culture of the research participants’ organisations through “prolonged engagement” (Shenton, 2004, p. 65). Rapport and trust need to be established in order that participants are frank and honest in their answers to questions, rather than feeling pressured to give an answer that they believe the researcher is expecting. The risk of failing to give adequate consideration to the relationship between researcher and participants can undermine the basis of action research as an emancipatory process: “A self-critical account that situates the researcher at the centre of the text can perpetuate the dominance our emancipatory intentions hope to fight.” (Humphries, 1997, p. 4.10). Power relations between myself and the research participants are considered further in Section 4.5.

Credibility of the data: This can be established through “triangulation” of information (Shenton, 2004, p. 65), i.e. by using different methods for collecting data relating to the same phenomenon. Other provisions that can be made are the use of “iterative questioning” (ibid., p. 67), i.e. covering the same ground with rephrased questions in order to establish

consistency or contradictions in responses, and “member checks” (ibid., p. 68), i.e. presenting back data to research participants to check that transcripts, inferences and emerging theories agree with what they meant to say.

Credibility of the findings: This can be established by relating the findings to existing theories and previous research findings, and by opening up the research design and findings to scrutiny by peers and those overseeing the project. Whilst being critical of positivist notions of

impartial knowledge, action researchers recognise the importance of engaging in debate with fellow academics advocating more conventional methodological stances (Brydon-Miller, et al., 2003; Levin, 2012). Presenting findings to other researchers from similar and contrasting research perspectives enables the researcher to consider “alternative explanations” (Levin, 2012, p. 143). Consideration of alternative interpretations of the data from different

perspectives should be seen as lending credibility to the findings, rather than as a weakness, provided that sufficient consideration is given to these different perspectives:

“When the readers’ different perspectives on a text are made explicit, the different analyses should also become comprehensible. Subjectivity in this sense of multiple perspectival interpretations will then not be a weakness, but testify to the

fruitfulness and the vigor of interview research.” (Kvale & Brinkmann, 2009, p. 213)

To promote transferability, the researcher must provide sufficient contextual information in the research report to enable the reader to make an informed judgement about the extent to which the findings can be related to his or her own situation. One way of doing this is by providing “thick description of the phenomenon under scrutiny” (Shenton, 2004, p. 69), which should include detailed information on who was included in the study, how they were selected and methods used for data collection and analysis. It is important that the reflexivity and the critical subjectivity of the researcher (see above) are made clear and transparent in the

reporting of the research. Also of relevance is the provision of contextual details relating to the research setting and the backgrounds of the researcher and research participants.

In order to achieve dependability, enough detail about the research process needs to be made available by the researcher to enable the study to be repeated by any future researcher who may wish to do so. The research report should include detailed descriptions of the “research design and its implementation, … the operational detail of data gathering … and reflective appraisal of the project” (Shenton, 2004, p. 71). Considering alternative explanations and interpretations of the data from different perspectives is important here, because the perspective of a future researcher may differ from that of the original researcher.

To establish confirmability, the researcher needs to convince the reader that the research findings are derived from “the experiences and ideas of the informants, rather than the characteristics and preferences of the researcher” (Shenton, 2004, p. 72). Many of the provisions for promoting credibility are relevant here including triangulation of information, reflexivity and critical subjectivity of the researcher. The characteristics of the research design and processes need to be reviewed carefully and periodically to ensure that the research is genuinely ‘participatory’ and ‘collaborative’ in nature.

These eight criteria detailed above constitute the framework I use to ensure the

trustworthiness of my research. The table below summarises how the framework is applied to aspects of my research design (see Chapter 4).

Table 1: Application of framework for ensuring trustworthiness: Criteria for

ensuring

trustworthiness:

Aspects of the research design:

Credibility of the research

processes

• Processes are derived from those used successfully in comparable research projects, e.g. empathetic interviewing, participatory action research, methods drawn from grounded theory.

• Secondary analysis/categorisation of the data focusing on research processes.

Credibility of the researcher

• Use of ‘autoethnobiography’ (see Sections 2.3, 2.5, 3.1).

• ‘Reflexivity’ through maintaining a reflective journal and ‘code log’. Credibility of the

relationships between researcher and participants

• ‘Prolongued engagement’ through selecting those I have worked with in the past to become teacher researchers.

• Previous experience of working collaboratively with teachers.

• Previous knowledge of, and engagement with, teaching mathematics for social justice.

• Transparency and openness regarding the aims and processes of the research.

• ‘Empathetic’ approach to interviewing. Credibility of the

data

• ‘Triangulation’ through data from research group meetings,

interviews, short reports, student surveys and research journal field notes (both mine and those of teacher researchers).

• ‘Iterative questioning’ through following up responses, made during research group meetings, in interviews.

• ‘Member checks’ through presenting analysis of data back to teacher researchers for verification and further comment.

Credibility of the findings

• ‘Plugging in’ the data to theories (see Section 3.7).

• Presenting and discussing findings regularly to/with other researchers at informal meetings, seminars, conferences and ‘Special Interest Group’ meetings.

• Considering alternative interpretations of data analysis from different perspectives.

Transferability _• ‘Thick description’ of context through ‘autoethnobiography’, providing details of research design and detailed case study of teacher

researchers, research group and research model.

Dependability _• Providing details of research design, in particular methods of data collection and analysis.

• Reflective evaluation of the research project. Confirmability _• ‘Triangulation’ and ‘reflexivity’.