Credibility and dependability - a comparative overview of the data analysis

4.6.1 Dependability

In order to ensure a good level of dependability of the methodology in this research project, the pilot study was carried out. This served the purpose of ensuring that the methodology would result in data that were sufficient to reveal the conflict among BSL users in the workplace before moving to collecting the further data for the thematic analysis. In order to test the consistency of the research method, the pilot study activities were carried out using the same format that would be used for the data collection for analysis later, in order to form a replica study; examples from the pilot study, which confirmed that the data collection method was effective, are presented in section 4.5.1.3 above. The use of a pilot study, then, served to ensure typicality, as the issues that were raised during the pilot discussions were able to be compared with the results of the actual data so different participants were used for the extended study.

4.6.2 Triangulation

Many research scholars have questioned traditional concepts of qualitative and quantitative research methods, and a diverse range of approaches are now used to provide robust checks during the research process, such as multi-strategy research methods (Bryman, 2004), mixed methods (Creswell, 2009; Taskhakkori & Teddlie, 2003), and multiple methods (Mertens, 1998). Scholars continue to find challenging ways to validate their research, and triangulation can be an effective method, depending on the nature of the specific thesis or research study in which the findings are to be validated.

This modern notion of triangulation involves conducting several diverse junctures of data collection and providing explanations for why each is used and how it divulges or opens up information about the data to us. The central reason for adding triangulation to a study, then, is to achieve or gain the credibility of the research outcomes that is crucial. This is concisely referred to by Mertens (1998: 183) as the “check on factual data”. Denzin (1970; 1978) refers to several triangulation designs, and proposes a process of ‘methodological triangulation’ for research. Furthermore, Patton (1999)

161

categorises four types of triangulation: methods triangulation,triangulation of sources, analyst triangulation and theory/perspective triangulation. Further options include Kirk and Miller’s (1986) time triangulation, space triangulation, level triangulation, theoretical triangulation and observer triangulation. Other triangulation methods, which are of more interest to this study, include the validating quantitative data model, the data transformation model and the convergence model put forward by Creswell and Plano Clark (2007). For this research, elements of the data transformation model are used, as the findings are transformed into some numerical data, and a process of

‘methods triangulation’ serves to validate the findings. The triangulation process begins with the use of the pilot project (see section 4.5.1), carried out in order to test the research activities and achieve credibility and dependability. The pilot stage of the study enabled a flow of data with which to ensure the dependability of information from the qualitative group discussions and the individual self-testimonies, and between the pilot and the extended study activities, and is illustrated in Figure 4:13 below:

Methodological Triangulation

Figure 4-13 Data Triangulation Methods Segregated

QGD Group

MIxed QGD Group

Triangulation

Self Testimonies

162 4.6.3 Credibility

Alongside the robust checking of research data findings that triangulation provides, the credibility of the results is an important aspect of the research validity process. Lincoln and Guba (1985) propose a four-element process of additional validity, which supports the testing of the data: credibility, transferability, dependability and confirmability (p.

315). A rigorous, multi-level process such as this ensures that the research findings are not only credible, but can also be transferred to other contexts and remain dependable.

Lincoln and Guba suggest that the trustworthiness of a research study is important to evaluating its worth and the four elements provide the following:

Credibility - confidence in the 'truth' of the findings;

Transferability - showing that the findings have applicability in other contexts;

Dependability - showing that the findings are consistent and could be repeated;

Confirmability - a degree of neutrality or the extent to which the findings of a study are shaped by the respondents and not researcher bias, motivation, or interest.

(Lincoln & Guba, 1985: 315)

In the case of this research project, it was necessary to verify the allocation of the extracts taken from the data to the assignment of sub-themes (presented in section 5.2.2). In line with Merten’s (2005: 259-260) notion that peer debriefing is often an appropriate way to achieve research credibility, a suitable academic peer, who is a Deaf, native sign language user from a family with many generations of Deaf people, was selected to perform a peer debriefing of a selection of the extracts. The selection was made on the basis of one extract from each sub-theme being chosen.. This resulted in 20 extracts being checked. The peer debriefer was provided with a paragraph explaining each sub-theme, and the extracts with the sub-themes that they had been assigned to.

The debriefer read the paragraph to understand the sub-theme and then checked to see if he agreed that the extract was assigned to the appropriate sub-theme. The peer

163

debriefer agreed that the allocation of each extract to its sub-theme was appropriate.

The peer debriefer was sent the 20 extracts from the translated transcriptions and the sub-themes that they had been allocated to by email. This process aided the credibility of the research by checking that the first stage of the thematic analysis was effective and served to minimise the researcher’s sole judgement of the participant contributions.

The extracts selected for peer debriefing are presented in the following table:

Table 4-5: Peer-debriefing Data Extracts

Themes Sub-themes

Audism

Assimilation Oppression Isolation Extract 32: IST 1 Extract 17: DBSLU 4,7 Extract 13:

DBSLU 2 

Attitude

Deaf people Sign Language Hearing people Reaction Extract 39: IST 3 Extract 12: HBSLU 5,6 Extract 46: D/HBSLU 7 Extract 10: D/HBSLU

2,5

Paternalism

Authority Passivity Cultural Awareness Benevolence Extract 71: IST 1 Extract 23: DBSLU7 Extract 15: DBSLU 7 Extract: 65 D/HBSLU

1,7,4

Transition

Identity Policy Incongruence

Extract 21: DBSLU 4 Extract 5: HBSLU 6 Extract 54: IST 6 

164

With regards to the remaining levels of the research credibility process, transferability of the finding of this study can only be achieved when future research has studied the same language user group in other contexts. Previous research, however, has shown that conflict exists among Deaf and hearing people in general (e.g. Young & Temple, 2014) and research such as the study carried out by O’Brien (2005) has evidenced tensions in interrelationships among Deaf BSL users and hearing non-signers in a social setting. There is evidence to a certain extent, then, that the findings of this study have applicability in other contexts, and this is at least some step towards checking for transferability. Due to the absence of research with a group of Deaf and hearing participants who are all BSL users, the pilot study was an important aspect of this project in terms of dependability, and serves to show that, while the experiences may be expressed in a different way, the topics of discussion are consistent when the activities are repeated, as described earlier in this section. Interestingly, despite the difference in participants recruited for the two data sets, the pilot participants and the extended data participants expressed much the same experiences. The recall of the experiences was expressed in slightly different ways but the themes to be drawn from them would be very similar. For example, in the pilot data, participant D/HBSLUA states:

I found that even if you had the very best policy which could all the different eventualities it still didn’t change the mindset and attitude of those with a very stubborn attitude towards sign language. My experience involves working in large mental health hospitals with strict policies but it doesn’t always affect the change of attitude required by some hearing people. I remember when I was about 18 or 19, I thought the policy was very good but the policy itself didn’t solve the attitude issues with some people (pilot data, extract 2).

This is comparable to participant DBSLU7, who also refers to the policy not having the desired effect:

I have spent untold hours telling hearing people to please sign and to respect the rules of our communication policy. Instead of concentrating on my work and what I was supposed to be doing I was getting all stressed and worked up over having to persistently tell people to sign and not speak (extended data, extract 22).

165

That is, the themes are consistent when the same researcher repeats the analysis.

However, another researcher conducting the same analysis with the same data set may draw very different interpretations of the experiences, and may come to somewhat different results. As Rodgers (2014) notes,

“…a different investigator might elicit different descriptions from people…changes in context for data collection, and other factors that can influence what is obtained in a qualitative study…The question therefore is not whether the same results would be obtained if someone else did the research.

The appropriate question is whether the results obtained by the researcher who did the study are appropriate, reasonable, and the processes can be traced and documented so that the results are defensible."

(Rodgers, 2014: 183)

As explained in section 4.4.2, the qualitative group discussion participants were left to discuss the topic without the researcher’s presence, and this helped to achieve a level of neutrality within the data collection and analysis phases of the project. This minimised the subjectivity level and provided confirmability for the study, helping to minimise the researcher’s judgement during the data analysis process and ensure that the data was shaped by the respondents and not by researcher bias, motivation, or interest.