Reliability and Validity: Trustworthiness

Qualitative research methodologies are sometimes criticised in the quantitative social science literature for their lack of standardisation and systematic data collection and analysis techniques (Ten Have, 2004). According to Bryman (2008), quantitative literature suggests using qualitative methods only for exploratory studies, because these methods do not have

reliable and valid foundations. However, it is actually possible to adapt these foundations to

qualitative research, or develop alternative criteria specially designed for qualitative research. Exploring reliability and validity considerations, this section explains how ‘trustworthiness’ can be applied as a criterion for assessing qualitative studies.

Reliability is concerned with the degree of consistency of research findings (Saunders et al.,

2009). Findings are reliable if they are consistent over time, accurate representations of the population, and replicable under similar methodologies (Creswell, 2009). Validity refers to the issue of truthfulness of research measures. It is concerned with whether the findings are really about what they claim to be about (Saunders et al., 2009). In other words, valid measures successfully assess the concepts that they need to measure, in order to deliver conclusions. Since positivism is based on experimental methods and seeks generalisations, it is necessary to assess and ensure the quality of research by using reliability and validity criteria. However, their relevance to interpretivist and qualitative research is in dispute (Bryman, 2008).

105

Stenbacka (2001) claims that qualitative research requires its own quality assessments. She argues that since reliability is about measuring and the purpose of qualitative research is to generate understanding rather than measuring, reliability is irrelevant to the assessment of qualitative studies. Other researchers have claimed that qualitative research should be judged by the same criteria as quantitative research, and have tried to adapt the concepts of reliability and validity to qualitative research. For example, LeCompte and Goetz (1982) adjusted reliability and validity concepts to qualitative research by changing their meanings. According to their terminology, reliability (also called here internal reliability) can be maintained by having more than one observer. If all of the researchers/observers agree on the explanation of their data, the consensus of the group secures the internal reliability. On the other hand, (internal) validity measures the degree of match between the researcher’s observations about reality and the theoretical ideas that they develop before the data analysis (LeCompte and Goetz, 1982). Here, the central task is to structure the measurement in a way that reduces ambiguities by eliminating the possible alternative interpretations of findings (de Vaus, 2009)

A third position argues that qualitative studies have completely different perspectives to quantitative ones, therefore require different criteria which should be particularly tailored for each paradigm. Lincoln and Guba (1985) propose a version of the criteria to assess qualitative studies, which draws from the constructionist approaches. Their criteria are based on the belief that there is not only one absolute reality in the social world. Therefore, they deny the use of one simple method of assessing qualitative research: different studies require different criteria. Lincoln and Guba (1985) use the term ‘trustworthiness’ for their perspective that features specific characteristics of different research studies. Trustworthiness aims to maintain the quality of qualitative research through four separate criteria: credibility, transferability, dependability, and confirmability (Lincoln and Guba, 1985).

106

Credibility: Credibility is concerned with ensuring the acceptability of research findings. Interpretivist and constructionist studies promote the possibility of multiple constructed realities, so there is no one concrete interpretation of reality (Hirschman, 1986). According to Lincoln and Guba (1985), the researcher should determine the acceptability of data through adequate representations of multiple realities and credible reconstructions of original multiple realities. This can be ensured by good practice and member validations (i.e. submitting research findings to the individuals upon whom it is based and seeking confirmation) (Lincoln and Guba, 1985).

Transferability: Transferability refers to the degree to which findings can be used across other social contexts (Lincoln and Guba, 1985). Interpretative researchers do not tend to generalise findings, because no two social contexts are identical and objects under study have certain characteristics that illustrate the uniqueness of the context (Hirschman, 1986). Moreover, even the same research will not necessarily hold in the same context at a different time; it is always possible that different results may be revealed (Lincoln and Guba, 1985). Therefore, the results of qualitative research are based on context, the theoretical sensitivity of the social phenomena, and the skills of the researcher. This means that findings cannot be generalised beyond the range of study (Carson et al., 2001). Lincoln and Guba (1985) argue that since generalisability of findings is not possible, qualitative research should provide thick descriptions (i.e. rich details about the specifics of the context) to present contextual information. Thick descriptions allow researchers to make their own judgments about the transferability of the findings, so resolving problems with transferability individually (Guba and Lincoln, 1994).

Dependability: In order to evaluate the consistency of instruments of interpretivist and constructionist approaches, Lincoln and Guba (1985) proposed auditing by multiple

107

researchers. The use of multiple investigators enhances the dependability of the research process, because comparing the interpretations of different researchers helps to determine the elements that are consistent across multiple interpretations (Hirschman, 1986). However, this method is not widely accepted because it requires too much auditing effort due to the large data sets of qualitative studies (Bryman, 2008).

Confirmability: Confirmability ensures that researchers do not influence the conduct of the research and findings (Lincoln and Guba, 1985). Complete objectivism is not desirable for interpretative studies; the researcher should be involved with the topic of study and engage with it personally. The researcher’s interpretation should represent a set of conclusions designed according to their interaction with the social context (Hirschman, 1986). However, researchers should act in good faith and not allow their values and feelings to change the results or progress of the study (Lincoln and Guba, 1985). In order to ensure this, Lincoln and Guba (1985) again propose using external auditors who can review the documentation and confirm whether the conclusions are appropriate and supportable to the information collected (Hirschman, 1986).

Interpretivist approaches, contrary to positivism, aim to understand rather than measure. Constructionist approaches believe that research findings are always partial and differ for every situation (Crotty, 1998). Therefore, having interpretivist and constructionist approaches, it was decided to follow Lincoln and Guba’s (1985) framework of trustworthiness to assess the credibility of the current study. Member validations were sought to determine the acceptability and representation of data (credibility). For this purpose, four research participants were consulted at the end of the data analysis period for their consideration and comments by providing them with a summary of the findings. Moreover, thick descriptions were used to present contextual information to allow other researchers and readers to make

108

their own judgements (transferability). Chapter 2 introduced Facebook and discussed the structure of Facebook sections, aiming to provide an understanding of the context. Findings were presented (Chapters 8 and 9) with explanations and additional information where necessary to allow readers to understand the context. Quotations were also included to illustrate the findings. Finally, the dual supervision process aimed to evaluate the consistency of the study and appropriateness of conclusions (dependability and confirmability).

Additionally, Carson et al. (2001:67) believe trustworthiness can be ensured by following three principles at various stages of the study:

(1) “careful use, interpretation, examination and assessment of appropriate literature, prior theory and empirical results;”

(2) “careful justification of the methodologies employed in a study;”

(3) “and careful structuring of data analysis to ensure full and descriptive evaluation and assessment”

In order to maximise the trustworthiness of the present study, these three principles will be followed at all stages.

The only exception to the approach of trustworthiness in this study is regarding the transferability or generalisation of the research findings. Contrary to the view that interpretivism cannot make generalisations about social life, Williams (2000) argues that interpretivist approaches can, and in fact in most cases do, make generalisations. He claims that “generalisation seems to be inevitable in interpretivist research. Indeed, virtually every reported study will contain at least some kinds of generalising claim” (Williams, 2000:210). He suggests that the possible reason for the difference between his and other interpretivist researchers’ views may lie in the definition of generalisation. Believing the existing definitions of generalisation do not comprehend all possible meanings, he suggests three separate definitions for generalisation: ‘total generalisation’, ‘statistical generalisation’, and

109

‘moderatum generalisation’ (Williams, 2000). According to Williams (2000), total generalisations are impossible for social research, because in order to ensure a total generalisation, the sample of the study should be identical to the population in every detail. This approach is similar to Lincoln and Guba’s (1985) belief about generalisations. Secondly, statistical generalisation, happens when the probability of a situation occurring can be estimated by looking at samples. Even though this is only possible with quantitative data, in social sciences some systems lead to statistical prediction to varying degrees of success, so this type of generalisation might be possible. Since data and the context of interpretive research are not always suitable for statistical analysis, statistical generalisations are not always possible. Nevertheless, interpretivists can and do make ‘moderatum generalisations’, or generalisations of everyday life. This is defined as occurring “where the aspects of [sample] can be seen to be instances of a broader recognisable set of features” (Williams, 2000:215). Williams (2000) suggests that researchers attempt to interpret data according to a subjective frame of reference of their data source and social environment. In this way, they can explore the characteristics of each case in its particular context. Due to the cultural consistency in the social world, these characteristics are generalisable in their particular social context, so he concludes that moderatum generalisations are possible in interpretivist research:

“thus everyday moderatum generalisations are what it is that the researcher wants to

understand, and of course if she can understand them then she will know something of

the cultural consistency within which they reside and is then able to make her own generalisations about that cultural consistency” (Williams, 2000:220, emphasis added by italics)

Therefore, it can be concluded that following Williams’s (2000) proposition, this research aims to make moderatum generalisations about online consumer complaining behaviours.

110

6.6. Conclusion

This chapter has presented the philosophical standpoint of the current study. The exploration and review of theoretical perspectives suggested ‘interpretivism’ as the appropriate research perspective for this study. Interpretivism aims to reach deep understandings and detailed descriptions of the subject of research. In interpretivistic studies, there is no one true reality, instead individuals have different perspectives about their realities. Considering that the perceptions of reality can change in different contexts, the research findings are based on contextual understanding and interpretation of data. Moreover, in order to help the researcher to grasp participants’ perspectives of reality, interpretivist studies sometimes require researcher involvement with the research context. Constructionism, which views reality as being continually constructed by individuals and their interactions, addresses the epistemological and ontological questions of this study. In particular, reality is seen as individuals’ own constructs of their online identities and behaviours, and it is acknowledged that reality might change with different participants, researcher or context. Therefore, the research findings cannot be generalised. However, they can be used to reach ‘moderatum’ generalisations, defined by Williams (2000) as generalisations of everyday life: the findings of this research are generalisable in their particular context, and should be interpreted based on its subjective frame of reference. This research uses qualitative research methods which focus on understanding how and why a social phenomenon occurs in a particular context. Finally, in order to ensure systematic data collection and analysis, Lincoln and Guba’s (1985) research criterion, trustworthiness, was followed. The quality of the current study will be assessed by the four dimensions of trustworthiness- credibility, transferability, dependability and confirmability-. The next chapter builds upon this discussion by presenting the methodological considerations and the research design.

111

CHAPTER 7- METHODOLOGICAL CONSIDERATIONS AND