Mixed Methods by Integrating Case Study and Survey Research

Research

Should this study be a single design using only a case study approach or should it be a multiple design? Is survey research methodology appropriate? What is the difference between a case study and a survey? A case study is in-depth research, as explained in the previous section, on a particular subject that usually spans a long period of time. When doing a case study one often physically submerges one's self in their study. As denoted by the title of the research, the study aims to examine the engagement in WBL programmes by stakeholders. This could take a quantitative angle where questions like ‘how many?’,

‘how often?’ or ‘when?’ would arise or be a qualitative angle where ‘why?’, ‘how?’ or ‘how do they feel about it?’ questions would arise. According to Yin (Yin, 1994), the

and quantitative angles in this research. ‘Why’ questions will try to identify the reasons

for any issues and challenges of current WBL delivery whereas ‘how’ questions will try to solve the issues through recommendations. Klein, Nissen et al. (1991) argue that the case for combining research methods generally, and more specifically the case for combining qualitative and quantitative methods, is strong. Nonetheless, as observed by Diesling (1971, 5), research designs that extensively integrate both fieldwork (e.g. case studies) and survey research are rare. The emphasis is on the qualitative case study method and how it can complement more quantitative survey research. Kraemer (1991) identified that survey research while very useful, is greatly improved when used in conjunction with other qualitative research methods.

A survey is less in-depth. However, a survey allows for research into a much larger group of people, and is often applicable to larger society if the results have a small margin of error. This is quantitative over qualitative. Yet, for a survey to succeed in elucidating causal relationships or even in providing descriptive statistics, it must contain all the right questions asked in the right way. Kaplan and Duchon (1988, 572) suggest that "The stripping of context buys 'objectivity' and testability at the cost of a deeper understanding of what actually is occurring". Survey research is inflexible to discoveries (relatively poorer 'discoverability') made during data collection. Once the work is underway, there is little one can do upon realizing that some crucial item was omitted from the questionnaire, or upon discovering that a question is ambiguous or is being misunderstood by respondents. Essentially, the researcher should have a very good idea of the answer before starting a survey. Thus, traditional survey research usually serves as a methodology of verification rather than discovery. In this way, survey results would be able to verify the in depth details discovered in case study interviews.

The survey approach refers to a group of methods which emphasize quantitative analysis, where data for a large number of organizations/programmes are collected through methods such as mail/online questionnaires, face-to-face/telephone/Skype interviews, or from published statistics, and these data are analyzed using statistical techniques. By studying a representative sample of organizations/programmes, the survey approach seeks to discover relationships that are common across organizations/programmes and hence to provide generalizable statements about the object of study. However, often the survey approach provides only a "snapshot" of the situation at a certain point in time, yielding little information on the underlying meaning of the data. Moreover, some variables of interest to a researcher may not be measurable by this method (e.g. cross-sectional studies offer weak evidence of cause and effect) (Gable, 1994).

While case study approach can provide important insights and discoveries during the study, fieldwork is a poor method for objectively verifying hypotheses. This would not be an issue in the current study as hypothesis testing is out of scope. Attewell and Rule (1991, 313) suggest that "Traditional survey work is strong in areas where field methods are weak". Surveys can accurately document the norm, identify extreme outcomes, and delineate associations between variables in a sample. Vidich and Shapiro (1955, 31) highlight the relatively superior 'deductibility' of the survey method over field methods. They observe that "Without the survey data, the observer could only make reasonable guesses about his area of ignorance in the effort to reduce bias." Jick (1983, 138) suggests that survey research may also contribute to greater confidence in the generalizability of the results as opposed to case study which is not meant for generalisation.

Attewell and Rule (1991, 314) highlight the "complementarity between survey and case study approaches to studying information technology", stating that "each is incomplete without the other". Danziger and Kraemer (1991, 367) point out that survey research and case study have always been alternative rather than competing sources of evidence and ideas, and Kling (1991, 346), Gutek (1991, 322) and Bikson (1991, 323) suggest that it is always best to utilize several methods of data collection to adequately address the impacts of information technology. This is applicable for WBL as well which applies information technology extensively in the process of delivery.

Attewell and Rule (Attewell and Rule, 1991, 299) further suggest that "conventional survey methods, such as mail questionnaires and telephone interviews, are inappropriate for many of the issues we need to address, and that a multi-method approach is more effective". By looking at the times of the above reference it is understood that there were no advanced survey data collection methods exist at the time which have been replaced by online questionnaires and Skype interviews today but the point of multi- method approach is more effective remains true nevertheless. Bikson (1991, 327) suggests that this view is desirable in most areas of social research; especially in a newly emerging sub-field such as the study of Information Systems (IS) in organizations. Bikson further points out that the IS research whether in cross-sectional or case study designs, has relied on a mix of information gathering approaches including structured interviews, self- administered questionnaires, archival material, and observation. Kaplan and Duchon (1988, 571) suggest that "no one approach to information systems research can provide the richness that information system as a discipline, needs for further advancement".

In concluding their contribution to the Iowa Social Science Research Center (ISRC) survey colloquium, Danziger and Kraemer (1991, 367) state that, "Our attempts to analyse

and interpret the extensive URBIS (Urbis Social Planning and Social Research team) has database have underscored the value of multiple operationism in developing grounded theory about information technology impacts. The survey research data on the one hand, and our field interviews and observations on the other hand, have constantly been alternative rather than competing sources of evidence and ideas. Similarly, continuing interaction amongst the URBIS colleagues, each offering somewhat different field experiences and interpretations, has slowed but enriched our individual understandings. To the extent that it is feasible, those undertaking research on information technology impacts should address common questions and hypotheses with multiple modes of data and multiple methods."

Wynekoop (1992) suggests that quantitative 'micro-level' analyses should be integrated with qualitative 'macro-level' analyses, in order that the ways in which individual behavior impacts organizational phenomenon, and the ways in which macro phenomenon have effects through individuals, be explicated.

Jick (1983) observes there is a distinct tradition in the literature on social science research methods that advocates the use of multiple methods. This form of research strategy is usually described as one of convergent methodology, multi-method/multi-trait (Campbell and Fiske, 1959), convergent validation, or what has been called 'triangulation' (Webb et al., 1966). These various notions share the conception that qualitative and quantitative methods should be viewed as compliments rather than as competitors. Jick underscores the desirability of mixing methods given the strengths and weaknesses found in single method designs. Through the use of multiple methods the robustness of results can be increased; findings can be strengthened through the cross-validation achieved when different kinds and sources of data converge and are found to be congruent or when explanation is developed to account for divergence (Kaplan and Duchon, 1988, 575).

It is agreed, that given human limitations, individuals must specialize in a limited number of methods as per Klein, Nissen et al. (1991). As observed by Orlikowski and Baroudi (1991), three methods have tended historically to dominate IS research: survey, laboratory experimentation, and case study. The majority of IS researchers are well versed in one or more of these three methods. Given the aforementioned potential benefits of combining research methods within a single research design, it therefore makes sense to leverage this talent by proposing approaches to combine these methods.

Mixed methods is viewed by its proponents as the third methodological movement ((Doyle et al., 2009); (Leech and Onwuegbuzie, 2009)), and the above discussion is not a bid to get involved in a methodological debate as stated earlier, the only objective here is

to align this research study with a particular tradition and defend its strengths and try to make up for its weaknesses. Doyle, Brady and Byrne (2009) outlined the main purposes for conducting mixed methods research as: Triangulation, completeness, offsetting weakness of other paradigms, and explanation of findings.

Several reasons are given for embarking on mixed methods research by Bryman (2006) who categorised them using Greene, Caracelli et al's (1989) earlier classifications as:

 Triangulation: This is meant as a convergence of results from methods to

corroborate findings from each source.

 Complementarity: This was described as using findings from one method,

to clarify results from another.

 Development: Using results from one method, to inform the development of

the other method.

 Initiation: Trying to discover contradiction and new perspectives by

using findings from different methods of data collection.

 Expansion: Seeks to extend the enquiry, by using different data types and

methods at different stages of the research.

From Bryman's (2006) analysis, most of the studies gave reasons as complementarity, and then secondly expansion, followed by development as third, and triangulation as fourth. The current study sits in the areas of complementarity, and triangulation, because:

Complementarity - student feedback from online questionnaires was verified with the interview feedback from students and other stakeholders for similar type of questions/variables with the aim of expanding/adding to data, and also as the second phase of data collection, enabling a small number of students who were interviewed.

Triangulation - the results from quantitative and qualitative analyses are triangulated for final outcomes. Triangulation is achieved within the case study by using

multiple data collection techniques ‘to pick triangulation sources that have different biases, different strengths, so they can “compliment” each other’ (Miles and Huberman, 1994). It also serves another purpose as Yin (2002) claims ‘to collect information from multiple sources but aimed at corroborating the same facts or phenomenon’. Therefore, data from interviews, questionnaires and documentation analyses are able to achieve triangulation. As Pickard (2007, 95) further explains, survey research can include qualitative and quantitative research hence mixed methods was adopted.

Accordingly, since ‘triangulation’ and ‘complementarity’ amongst the different stakeholder perspectives are important, it is appropriate to consider ‘expansion’ as this is a

case study which yields in depth data and thus using a mixture of approaches gives that depth. All above examples from the literature iteratively justify the advantages of using mixed methods, in particular, mixing survey research with case study, in order to evaluate the effectiveness of WBL in depth from the stakeholder perspective in a particular area.