Research design quality: Reliability and Validity

The research findings’ credibility underpins the discussion of the research design and the related issue of its quality (Saunders et al., 2011). According to Yin (1994), essential criteria about the design of any research are assisted by validity and reliability. However, the use of the criteria of validity and reliability has been questioned mainly by qualitative researchers. In response, some continued to use the traditional terms of validity and reliability in qualitative research while some others formulated new terms respectively such as credibility and transferability instead of validity and dependability instead of reliability (Saunders et al., 2012). Independently of the label used, validity and reliability are key criteria for assessing the research quality either in qualitative or quantitative research studies and they are concerned with the question of whether is measured, what is thought is measured and whether the conclusions are consistent. In particular, in mixed methods research design one of the most frequently discussed criteria regarding the quality of research design is the integration of findings derived from both quantitative and qualitative analysis (Bryman, 2011; Creswell & Tashakkori, 2007). The following section describes the measures taken in the current study in order to assist both the reliability and validity of the research design.

4.6.1 Data Triangulation and Methodological Triangulation

It is argued that in research, triangulation is best described as the bridging and the combination of multiple sources of data, of methods and perspectives in the study of a single phenomenon to confirm the conclusions derived from the data and create a single construct. Further, it is highlighted that is valuable as a strong means of ensuring rigor according to researchers who have been involved with mixed methods research (Yu, 2004). Triangulation is concerned with the use of different techniques of data collection in any single study and according to Saunders et al. (2009, p. 146) “it ensures that the data are telling you what you think they are telling you”. Yin (1994) argued that, in the case of the data collection process, triangulation is realized when data is collected from numerous sources. These sources can very well be actors from different participant levels within the research setting. According to the number of data collection sources this study has employed different sources, such as literature and official documents regarding the Greek higher education sector and the attempted accounting reform. Also, participants from different levels within the Greek HEIs have been extensively employed. In particular, Heads and Directors of the financial departments of the Greek HEIs have been surveyed or interviewed through a structured or a semi structured questionnaire respectively. Regarding the data collection process, multiple methods have been used such as an initial literature review and official document providing valuable content data, a sequential quantitative survey providing descriptive numerical data as well as a structured questionnaire survey providing quantitative data and finally, interviews providing appropriate data to be analyzed qualitatively and quantitatively. The results of the literature content analysis proved very useful in the development of the questionnaire which was administered in the survey process (empirical study). The

results extracted from the survey study were then utilized in the process of informing the semi-structured questionnaire used in interviews. Finally, in the interpretation phase, the findings resulted from each inquiry phase were integrated in order to be confirmed and corroborated in a smooth way. Consequently, data triangulation has been evidenced at each level of the inquiry process. More specifically, the evidence used was in multiple forms and derived from diverse sources. Thus, is clear that this process assisted in the establishment of construct validity and the reliability of the current research, as it clearly followed Yin’s (1994) principles for the collection of data.

4.6.2 Validity

According to Saunders et al. (2009), validity is concerned with whether the findings that appear as a result of the research are real. As it derives from the research literature qualitative researchers have utilized eight strategies concerning validity namely: prolonged engagement and persistent observation in the field, triangulation, debriefing, refinement of hypotheses as the inquiry advances, clarification of researcher bias from the outset of the study, solicitation of the views of the participants regarding the credibility of the findings and interpretations, rich and thick description and assessment from external audits’ (Creswell, 2007). Further, the author (Creswell, 2007) suggested that at least two of the above strategies be employed. As it has already been mentioned, the strategy of triangulation strategy has been employed for validity purposes in the current study. Furthermore, the researcher was fortunate enough since his supervisors were keen on checking and discussing the results with him. Finally, quantitative and qualitative data collection and analysis is analytically presented and explained by using both quantitative and qualitative techniques.

4.6.3 Reliability

The replication of results by other researchers who investigate a similar topic with the use of similar procedures increases reliability (Yin, 1994). Reliability is associated very closely with the techniques used for the data collection and analysis gaining thus consistent findings (Saunders et al., 2009). This implies the need for strong documentation of the research process and evidence of linking the conclusions and findings. In the current research, an analytical and detailed presentation and explanation regarding the research design and process has been attended. The research design was strongly related to the literature relative to methodological issues in social and organizational studies and the investigation of complex social phenomena such as the MA change in particular settings. Furthermore, the research methods of data collection and analysis that have been employed for the scope of this study have been presented in details. Finally, the following chapters describe in detail and report the findings and results derived in each phase of the research. Appendices also present the structure of the questionnaire employed in the survey and the interview schedule, as well as, the semi structured questionnaire. In the final chapter, an overall analysis of the entire research is developed by presenting, integrating and harmonizing the results in order to be corroborated/confirmed and allow for a complete and more comprehensive understanding of the investigated phenomenon.