Validity and reliability, in their traditional sense, are associated with quantitative epistemology. Scholars have argued that such criteria to judge research 'quality' are problematic for research based on a different worldview. Bassey ( 1 999), for instance, argues that "a case study is a study of singularity which is chosen because of its interest to the researcher (or the researcher's sponsor) . . . . It is not chosen as a 'typical' example in the sense that typicality is empirically demonstrated, and so issues of external validity are not meaningful" (p. 75). Hence these terms are said by some to be unsuitable for qualitative research. Many scholars call for a new set of criteria to determine 'quality' in qualitative case study research. For Janescik ( 1 994) validity "has to do with description and explanation, and whether or not a given explanation fits a given description. In other words, is the explanation credible? (p. 2 1 6). Similarly, reliability is viewed as "a fit between what they (qualitative
Chapter Three: Methodology 1 02
researchers) record as data and what actually occurs in the setting under study, rather than literal consistency across observations" (Bogdan & Biklen, 2003, p. 36). Lincoln and Guba (1 985) argue that when a dialectical methodology, oriented towards the production of interpretation is employed, concepts such as validity, reliability and objectivity, should be replaced with trustworthiness and authenticity. This study was concerned with issues of trustworthiness. Authenticity, which deals with a set of issues concerning the wider political impact of the research, such as fairness and agency, was deemed less applicable to this study. Lincoln and Guba (1 985) further translate trustworthiness into credibility, transferability, confirmability and dependability.
3. 8.4. 1 Credibility
Establishing credibility of research findings involves ensuring that the research was conducted according to the canons of good practice. For Bryman (2001 ) credibility is equivalent to internal validity in quantitative research. Credibility can be achieved by the congruence of results from the many information collection techniques used by qualitative researchers (LeCompte & Goetz, 1982). This may include interviews, participant observations, checking transcriptions and other information with participants. Submitting initial interpretation and information analysis back to the participants is seen to be important for confirmation that the investigator had correctly understood the social world (Bryman, 200 1 ). Bryman notes the use of two techniques for achieving credibility: respondent validation and triangulation.
Although analytical interpretation and conclusions were not sent back for respondent validation, all transcriptions were sent for endorsement from participants. In this study credibility was dealt with mainly by employing triangulation. Triangulation is defined as the employment of a number of methods to analyse a study with the hope that the methods result in data collaboration and congruence (Denzin, 1 970). Denzin and Lincoln (2000) suggest triangulation at the level of data, method, investigator and theory.
In this study, triangulation was undertaken at the level of data and methods. At the level of data sources, data were triangulated between policy and other documents collected from the field and interview transcripts and international literature on QA in higher education. At the level of methods, triangulations were achieved between documentary analysis, interviews and survey analysis.
Chapter Three: Methodology 1 03
3. 8.4.2 Transferability
Since qualitative case study research entails the in-depth study of one or a few cases, findings from such research tend to be contextually bound. Stake ( 1 995) also observes the difficulty of generalising findings across social settings and states that "case studies seem a poor basis for generalizations . . . . The real business of case studies is particularization" (p. 86). However, Guba and Lincoln ( 1 985) argue that transferability of research findings can be achieved through production of what Geertz (as cited in Bryman, 200 1 ) calls thick descriptions. Rich and thick descriptive accounts of the context for the study and the individual cases can enable others to compare situations elsewhere and make judgments. If there is to be transferability, however, "the burden of proof lies less with the original investigator than with the person seeking to make application elsewhere" (Lincoln & Guba, 1 985, p. 298).
This study attempted to provide as much detail and description as possible, relating to the three contexts of study and the QA policies and practices in the three case study sites.
3. 8. 4.3 Dependability
Dependability is the substitute criterion for reliability in quantitative research. As replication, in the traditional quantitative research sense, is not appropriate in qualitative research, Lincoln and Guba (1 985) propose the adoption of an audit process. This involves keeping complete records of all phases of the research process, including research problem, selection of participants, field notes, interview transcripts, data analysis decisions, etc. in an accessible manner (Bryman, 200 1 ). An independent auditor or a peer would be able to examine the research process to establish the extent to which proper procedures have been followed.
In this study, issues of dependability were managed by keeping a purposeful and reasonable record of all phases of the research process. This included the following: different research proposals; interview schedules; interview tapes and transcripts; field notes; minutes of meetings with supervisors; letters seeking permission and approval to undertake research in the institutions; official research visa/permit to Fiji; and statistical data gathered from the three case study sites.
3. 8. 4. 4 Confirmahility
Objectivity, as achieved in quantitative research, is not possible in the same way in qualitative research. However, as Guba and Lincoln (1 985) argue, it is important to ensure that the researcher has not overtly allowed personal values to sway the conduct of research and consequently the findings of an enquiry. This can be achieved through establishing confirmability.
Confirmability in this study was established by triangulation of data and method.
3.9 Conclusion
This chapter has described the methodological approach taken to consider the key research question: How does the newly emerging higher education system in the Maldives assure quality amidst a rapidly globalising QA policy environment?
This key question, translated into three principal research questions, was considered within a multi-site case study. An intensive in-depth primary case study was undertaken in the Maldives, whilst two less intensive secondary case studies were undertaken in New Zealand and at the USP. A multi method approach, including interviews, documentary analysis, survey questionnaire and ' condensed' participant observation were used to gather data. Analysis was shaped by the framework of a globalising QA policy environment and educational policy analysis. Both manual methods and NVivo software were used in data analysis. Measures taken to achieve quality of the research included steps to ensure trustworthiness and authenticity. Trustworthiness was ensured through steps taken to establish credibility, transferability and dependability.
4.1 Introduction
Chapter Four
THE STUDY CONTEXTS
This chapter provides a brief background to the local contexts and the main quality assurance (QA) developments in each of the three case study sites. A more extensive review of the New Zealand context is undertaken, since New Zealand has been a leader in qualifications and QA reforms in the world. Hence, it has more experience to offer, in terms of useful lessons to other systems at early stages of development in QA. New Zealand also provided a unique example of a situation where external neoliberal influences and managerialism has provided the ideological basis of the QA reforms. The other two sites of study were relatively new to formal QA in higher education. The chapter is divided into three parts. Part one provides the New Zealand context. Part two presents the USP context and part three provides the Maldives and the MCHE context of QA development.
Part I: The New Zealand Context
The New Zealand case study was undertaken in one of the 1 9 Institutes of Technology and Polytechnics (ITPs) in the country (ABC College, 2007b). It is a regional polytechnic and it is identified by the pseudonym of ABC College. The institution's QA systems operate within the national policy and system for QA. Firstly, a brief examination of the essential external influences on QA in New Zealand higher education (or tertiary as it is more commonly known in the New Zealand context) is provided followed by an overview of the current QA system. The last section of this part provides a brief description of the ABC College.
4.2 The Macro (external) Influences on Quality Assurance in New
Zealand
Many of the New Zealand reforms in tertiary education, including those specific to QA, are the product of the reforms that had their genesis in the broader public sector reforms, which began in the 1 980s. The Fourth Labour government, elected in 1984, has been identified as being responsible for bringing about revolutionary changes within the entire education landscape of New Zealand. These substantial reforms were
Chapter Four: The Study Contexts 1 06
based on policy advice to the government, that was heavily influenced by key bodies of administrative and economic theory (see Middleton, Codd, & lones, 1 990; Olssen & Matthews, 1 997). Commentators identify four main bodies of knowledge that were particularly influential in shaping New Zealand's public sector reforms and subsequently the higher education sector as well. These were Public Choice Theory, Agency Theory, Transaction-Cost Economics and the New Public Management (NPM)14 (Boston, Martin, Pallot, & Walsh, 1 996). The Treasury was the chief institution that advocated these ideas and it was also responsible for translating them into specific policy proposals. Each of these main theories will be briefly considered in the following sections, in order to examine how they have influenced and impacted on tertiary education QA reforms in New Zealand.