Limitations

Flyvbjerg (2006) presents a robust defence of case study research against five common misunderstandings that are frequently levelled against it. Many of these critiques arise from conceptions of what research is and how it contributes to understanding that are rooted in positivist approaches. In his article, Flyvbjerg, defends case study research against the following misunderstandings:

- General, theoretical knowledge is more important than concrete practical knowledge,

- One cannot generalise on the basis of an individual case,

- The case study is not useful for generating hypotheses,

44 - It is often difficult to summarise and develop general propositions and theories on

the basis of specific case studies.

(Flyvbjerg, 2006, p40) Cohen et al. (2011) discuss in detail the issue of generalisation in case studies and highlight that this is a challenge that is often levelled at case studies which frequently use purposive sampling which is not statistically representative. This is discussed by Yin (2014 p40) who argues against attempts to make statistically based generalisations and promotes “analytical generalisations” which either refer to existing theories or seek to raise concepts which arise from the study, this point of view is echoed by Rule & John (2015) who make the suggestion that case study should focus on the specifics of the case but it is relevant to make tentative generalisations, likewise Stake (1995 p85) argues that “case studies are undertaken to make the case understandable”. In a similar vein, David (2007) advocates the suitability of case study in situations where the knowledge gained is intended to be used in some way with the proviso that it relates to the case only. However, a slightly different approach is promoted by Thomas (2013) who refers to the work of Stenhouse (1980) and points out that although it may not be possible to generalise from any given case study, the accumulation of data over time will build value from case studies. This echoes the point of view expressed by Stake (1995 p74) that it is from the aggregation of instances that understanding is built. A further perspective is offered by Van Wyhnsberghe & Khan (2007) who suggest that case studies should lead to working hypotheses or a collection of lessons learned, this is similar to the point of view that is presented by Harland (2014) who makes the point that case study is not attempting to replicate the scientific method and that it is up to the reader to learn from the study by reading from a critical perspective. It is these final viewpoints that will guide this study, that the aim will be to learn lessons for the context of the course in question and to offer the findings to a wider audience with the expectation that they will critically consider whether it has implications for other settings, this point of view is echoed by Hyett et al.

(2014) who state that case study is inherently comparative and does not seek to generalise to populations.

Other aspects which act as limitations to case studies are threats to validity. Yin (2014, p45) and Cohen et al. (2011, p295) discuss construct validity, internal validity, external validity and reliability and offer a critique of applying tests of these which stem from the scientific method or positivist approaches to research. On the other hand, Stake (1995 p108) does not explicitly refer to threats to validity, reflecting his interpretive standpoint, instead, he

45 discusses the need for triangulation of data sources and the relationship between the depth of data and the contestability of any claims based on it with more contestable claims requiring a greater depth of data. Yin (2014) and Cohen et al. (2011) suggest how threats to validity might be addressed in ways that are relevant to this form of research.

46

Tests Case study tactic

Construct validity - Use multiple sources of evidence - Establish chain of evidence

- Have key informants review draft case study report Internal validity - Do pattern matching

- Do explanation building - Address rival explanations - Use logic models

External validity - Use theory (in single case models) Reliability - Use case study protocol

- Develop case study database

Yin (2014, p45) Table 2: Yin's (2014) design tests

The use of a variety of sources of evidence which have been selected in order to illuminate key elements of the research question will provide triangulation (Stake, 1995) and to maximise construct validity (Yin, 2014). However, the approaches of Yin and Stake are harder to reconcile in other aspects, for example the use of case study protocol suggested by Yin (2014, p45) runs counter to the approach of Stake (1995, p72) who acknowledges that case study researchers make use of protocols but need to fall back on intuitive approaches when faced with situations that have not been previously encountered.

Cohen et al. (2011) discuss this and highlight the importance of the chain of evidence due to its role in allowing the reader to track through the process and judge its validity for

themselves. Comparable points are made by Fidel (1984) who argues for clear discussion of data such as interviews, or Harland (2014) who advocates high quality case study research by bringing the reader as close as possible to the experience in order to offer a believable insight, a similar comment is made by Hyett et al. (2014). Whilst such guidance is helpful, it is not always possible to achieve. For example, Yin’s (2014) tactic of having key informants review drafts or Eisenhardt’s (1999) suggestion that multiple investigators should work on data. Whilst these are not possible, this study will aim to increase construct validity by requesting that participants review the data that they have provided even if it will not be feasible for them to review the analysis of the data.

It is worth noting the comment that Hyett et al. (2014) make that Yin (2014) views case study in post positivist paradigm and thus his approach is to develop protocols for the researcher to follow. This is in contrast to the social constructivist perspective of Stake.

47 Likewise, Fidel (1984) argues that case studies should not be rigorously planned as the researcher should be able to react to what they find.