Rigor and validity of the study

Apart from its increased popularity, the use of mixed methods was widely criticised in the 1990’s, particularly by constructivists or interpretivists, who conceptually rejected the idea of positivism in social science research. Other researchers still favour more traditional ways in conducting research projects either by following the rules of quantitative or qualitative paradigms alone. Within social science its researchers such as Lincoln and Guba (1985) and Schwarndt (1989) believed that qualitative and

145

quantitative approaches are incompatible, despite the wider use of mixed methods. Creswell and Plano Clark (2011, p.p. 13-15) highlighted the challenges of using mixed methods such as the skills possessed by the researcher, the availability of time and the resources and the ability to convince others. In addition to these, they list eleven key controversies and questions being raised in mixed methods (2011, p.37). One of them is related to the discourse of mixed methods; who controls the discourse and are the methods nearing a ‘meta narrative?’ Therefore, it is important to be confident of the quality of the data.

In all types of research, including mixed methods, it is essential to assure quality. Issues surrounding research quality have been discussed widely by many quantitative and qualitative scholars by using a generic term ‘validity’ for more than a quarter century (Rolfe, 2006) yet, no single consensus has been agreed. In contrast, validity in mixed methods is an area which is still in its development. Creswell (2012) defined validity in mixed methods as employing strategies that address potential issues in data collection, data analysis, and the interpretation that might compromise the merging or connecting of the quantitative and qualitative strands of the study and the conclusions drawn from the combination. He suggested the principles of maintaining the validity of both quantitative and qualitative strands in mixed methods. He also recommended using the term ‘validity’ due to the acceptance of this term by both quantitative and qualitative researchers, although, it has been well known that many qualitative researchers would argue against any measure of the quality of qualitative research by using quantitative strands. For example, Sandelowski (1993) argued that the validity of qualitative research should not be measured based on its ‘truth’ or ‘values’ as in quantitative research, but from it being ‘ trustworthy’ which makes it more visible and understandable by ‘leaving a decision trail’, which allows the readers to track the

146

research process in order to find out its quality. Previously, Lincoln and Guba (1985) clarified the meaning of ‘trustworthy’ by indicating four components: credibility (analogues to validity in quantitative research), dependability (reliability in quantitative research), and transferability (external validity in quantitative) and conformability (presentation).

As argued by Dubois and Loiselle (2009), it is important to address potential validity issues from participants’ recruitment, selection tools and data analysis. Murphy (2007) and Wilkins and Woodgate (2008) also highlighted that the design of mixed methods needed to be carefully structured, and with experience the researchers should be able to combine the approach more effectively (Patton, 1990); Reichardt and Cook (1979). The quality or validity of this PhD study was ensured by maintaining the elements of trustworthiness or validity as highlighted above. Creswell and Plano Clark (2011, pp. 240-241) stated that the measures of validity and reliability of mixed methods are currently under development. They suggested three areas which needs to be carefully monitored throughout the research process: data collection, data analysis and

interpretation. The following sections explain how these three areas were maintained throughout the research process

6.4.1. Data collection

The research participants in this study were recruited from 2 samples: DSNs and Patients. Purposive sampling method was deemed to be appropriate to meet the research needs. All seven nurses were employed as DSNs with experience in delivering diabetes consultations to adult patients. The patients recruited for the QUAL phase were carefully selected by considering factors such as age, gender and types of diabetes. In the QUAN phase, the sampling criteria were followed to recruit

147

the patients. These patients were adult, living with type 1 or type 2 diabetes, they did not suffer from any severe mental health problems and they had been seen by the DSNs within the last three months.

The data collection tools, that is, the interview schedules, the tape recorders and the Consultation Quality Index- 2 (CQI-2) (Mercer and Howie, 2006) were carefully prepared. The interview schedules were piloted before the tools were used to guide the interviews with the patients and DSNs. The audio recorders were tested by the

researcher a few times to make sure that they were sensitive and able to capture human voices. The questionnaires (CQI-2) had been previously used and validated in previous studies (Mercer and Howie, 2006) but in addition to this, a pilot was conducted prior to distributing questionnaires to the 150 patient respondents in this PhD study. In the pilot, the questionnaires were sent to 10 patients who were selected randomly.

Unfortunately, only 4 completed questionnaires were returned, but all questions were answered by the respondents and only minor amendments were made.

The internal consistency or reliability of this study was psychometrically tested by measuring the Cronbach’s (alpha). The first test measured the 10 items used to measure empathy. The Cronbach’s alpha was 0.971 and the Chronbach alpha-based on standardized items was 0.973 (an alpha of at least 0.7 is normally considered adequate). The Anova with Cochrane test shows ρ= 0.134 (NS). The second

psychometric test looked at the 6 items used to measure the consultation outcome. The Cronbach’s alpha was 0,949 and the Chronbach alpha-based on standardized items was 0.950. The Anova with Cochran test shows ρ= 0.667 (NS).

148

The above results show the reliability or internal consistency of the study, as in practice any Chronbach’s alpha test higher or equal to 0.9 is considered to be excellent.

6.4.2. Data analysis

There are many strategies to check the qualitative data within a mixed methods study. Creswell and Plano Clark (2011) suggested that researchers should use member- checking which they considered as the most common way. The transcriptions of the data in this PhD research were checked by two people including the researcher and a professional proof reader. The interview transcriptions were returned to the DSNs individually for comment. They were asked if the texts were accurate concerning the recording of their experiences. Some minor amendments were made following their feedback. The majority of the patients did not wish to check the transcriptions with the exception of one patient who was interviewed in his home; he did not make any amendments. Another way to check the validity of qualitative data involves

‘intercoder agreement in qualitative research’ (Miles and Huberman, 1994). In this activity, several people need to be involved in order to assign codes and to develop themes. Within this PhD project, the coding and themes were checked by the research supervisors. In addition to these, an oral presentation was delivered to the DSNs to disseminate and to check the preliminary findings. The DSNs agreed with the findings and expressed their interest in having the final results.

The data obtained from consultation observations were examined using conversation analysis. A short course in conversation analysis was attended to familiarise the researcher with the analytical process. Further assistance was obtained from other researchers familiar with this type of analysis.

149

The quantitative data was analysed statistically. The internal consistency or reliability was measured by calculating the Cronbach’s α (alpha). The overall inter-reliability of questions was 0,781, whereas the outcomes score was 0.949. In general, the α

Cronbach values of 0.7 to 0.8 are considered as satisfactory, although in clinical research the expectation of the values are higher (Machin, Campbell and Walters, 2007, p.209).

6.4.3. Interpretation

In exploratory design, Creswell and Plano Clark (2011) indicated the importance of this last stage. The researcher should be able to evaluate how the quantitative findings build or expand upon the qualitative results. In this interpretation stage, judgements need to be made whether the combined results answer the qualitative, quantitative and mixed methods questions.

Within this PhD project, the interpretation was closely monitored by the researcher and the research supervisors. In order to get feedback from a wider research

community, the preliminary findings were also presented at different conferences, including one mixed methods conference.