3.2 Case study research
3.2. 1 Issues in case study research
3.2.2 Rigour in Case Study Research 3.3 Action Research
3.3. 1 Action Research and the case approach
3.3.2 Stages of the Action Research Cycle
3.3.3 Community Action Research 3.4 The Checkland Methodology
3.4 . 1 Stages of the Checkland methodology 3.5 C hapter Summary
3. 1. Introduction
This chapter describes the methodology used in conducting this research. Chapter one outlined the aims of the thesis and chapter two reviewed the research literature relating to community informatics design methodologies. Section 2.3.2 of chapter two established that while there is a substantial body of research on community informatics, most of it consists of reports on the outcome of Community Informatics projects and the lessons drawn from these, but that there was little explicit consideration of the methodology used to conduct those projects.
The overall aim of this thesis is to add to that literature by establishing and
verifying a development methodology for use in Community Informatics projects. The process used to produce these guidelines was to first conduct empirical research in small scale applications, and then to apply the methodology produced by that research in full scale community situations.
The first part of this chapter presents an analysis of the case study method of research, and the second part describes action research, the methodology used in the community projects.
3. 2 Case study research
Few of the cases reported in c hapter two were based on any underlying theory. They simply used the available technology in an ad hoc manner as dictated by the technology itself. The research in this thesis uses action research in case situations to explore the principles underlying community informatics design.
In studying the interrelations between technology and the community of users, information systems research has turned to social science methods and paradigms, suitably adapted to fit the unique needs and circumstances of the community and the technology being applied (Leedy 1 993). Qualitative research methods work well for exploratory studies in new fields. Monitoring the progress of projects can be naturalistic and inductive allowing a holistic view of a dynamic situation (patton
1 990). As information technologies penetrate more and more into everyday life the social and psychological aspects of IT implementation have become more
important, and the purely technical aspects have become relatively less important: 'Cl proj ects are driven not by science, but by the values, ideologies, and political interests of the maj or stakeholders' (O'Neil 2002, p.77). Research in Information Systems has therefore adapted to take account of the needs of research being done in real world situations that are unique, non-repeatable, and open to observer bias. As a result there is a strong case study tradition in Information Systems research (Alavi and Carlson 1 992; Orlikowski and B aroudi 1 99 1 ). The underlying paradigm used is the 'natural science model' (Behling 1 980). However the natural science model is more suited to testing existing theories, than to formulating theories: the case study method is more appropriate when there is a need to study the
phenomenon in its natural setting; an emphasis o n the 'how' and 'why' questions; and a lack of previous studies or sound theoretical understanding (Benbasat,
Goldstein and Mead 1 987). The series of community based projects in this thesis fit that description. The research methodology therefore utilised case study research using the action research process.
3.2.1 Iss ues in case study research
Research is, in general, j udged on how well it generates new theories to explain observations, or tests existing theory against planned observations. For a theory to be accepted as valid scientifically it must satisfy four tests: it must be falsifiable; the various predictions extractable from the theory must be mutually compatible and not contradictory; the theory must be at least as good at explaining the observations as any other theory; and while falsifiable, the theory must actually withstand all attempts at falsification. Following Lee ( 1 989) the different
implications of each of these requirements for the natural sciences and case study research are discussed below:
Making controlled observations
I n the natural science model, the relationship hypothesised to exist between
different factors is tested by observing the interactions between those factors, while controlling for any confounding influences. In laboratory experiments this done by rigorously excluding extraneous factors; in social sciences it is done using control groups; in statistical experiments it is done by applying statistical controls such as regression analysis. In case studies, because it all takes place in a real world setting, and not in a natural laboratory, excluding extraneous factors is impossible; and examination of the case situation usually produces more variables than data points, effectively ruling out statistical control techniques.
Making controlled deductions
With mathematical propositions, making controlled deductions is axiomatic.
However, case studies usually do not lend themselves to quantitative interpretations so the researcher is seldom able to work with numerical data and use mathematical propositions. Instead the case researcher must manage qualitative data and extract meaning from language and behaviour. Making controlled deductions from qualitative data involves more uncertainty. There are fewer conventions to rely on than with quantitative data, and therefore more chance of drawing false
conclusions.
A llowingfor replication
Laboratory experiments are inherently easy to replicate, can confirm or extend particular findings and to thus ensure the objectivity of the research. However a case researcher is unlikely to ever meet the same conditions or events twice; the timing, situation, social structure, expectations and experience will vary between one case and the next. On the face of it, this makes it very difficult for a second researcher to independently verify the results by replicating the case.
A llowing for generalisability
The power of theories developed under the natural sciences model is their wide applicability: they hold true in a wide variety of situations. However the situations in case study research are generally unique and not replicable, and therefore would seem to be vulnerable to charges that they cannot be generalised to other situations. However, in this thesis the research output is a methodology for community
methodology in different cases should be the same, and able to be j udged
obj ectively. In so far as the outcome of different situations is a successful project then the tests noted above can be met. The next section discusses how these objections to the case study approach can be overcome.
3 . 2 .2 Rigo u r in Case Study Research
It is entirely possible to use case study research to satisfy the requirements of the natural science model. C ase study research involves the intensive study of a single case or series of cases, observing and recording the actions of individuals, groups, communities and their interaction with information technology and the proj ect methodology but it can be conducted within the scientific method. By following the procedures of natural science, case study research can be equally rigorous and falsifiable.
The core issues of controls, valid deductions, replication and generalisation might be thought to apply only to the case study research method but, in fact, the natural science model exhibits similar problems. Few propositions of science are directly verifiable as true: most concern unobservable entities such as molecules and chromosomes, or abstract concepts such as light waves or electricity. As a
consequence, the propositions of natural science are themselves verified by indirect rather than direct means. The basic sequence is to define the initial proposition, the theory which cannot be tested directly, and from this deduce or derive secondary propositions which can be tested directly. The test of a secondary proposition is held to be a test of the primary proposition. If the results of the test are false then the proposition that implies them must also be false. Controlled observation only comes in as the last step in the process.
Natural Controls in cases
By presenting alternative theories and applying those in the case situation, the researcher can compare the deductions or the predictions of each theory against observations in the context of the case. By comparing theories in the same case situation complete control over the confounding factors can be attained: the same confounding factors apply to both theories, therefore any differences must be due to the predictive power of the theories themselves.
In a community development case study the complex interaction between the individuals, the community, the technology and the development methodology
obscure the underlying dynamics, and any controlling phenomena must therefore
be inferred indirectly. While the unobservable phenomena cannot be tested for directly, competing theories can be used to derive predictions of what will be seen in the case situations, and so the theories can be verified or falsified.
Similarly, natural controls can be sought among the factors in the case. The researcher can choose predictions from the theory which are testable, and have a high likelihood of being seen in the case situation. In this way the researcher can take advantage of naturally occurring events to test one or more facets of the theories and give the opportunity of falsifying them or not. This type of control is commonly used in other natural sciences that are not amenable to laboratory experiments, astronomy and geology for example. This scanning of the environment for high likelihood events is not significantly different from the
statistician identifying independent variables in the data to use as statistical controls in multivariate regression analysis.
Controlled deductions in cases
Making controlled deductions in a qualitative data set is no different from the way they are made in quantitative data. In mathematical analysis the rules for deducing propositions are clear and easily checked by the principles of algebra. However, mathematics is a subset of logic, and it is the rules of logic which determine the rules of mathematical deduction. The same rules of logic apply to propositions stated in words: the formal logic governs what propositions can be derived, not the algebra. The natural sciences have a long history of utilising non-mathematical deductions, evolution and biology being prime examples.
Replicability in cases
It is seldom possible to replicate a case situation exactly or to apply the same theories in that same situation. However, using the theory comparison procedure, another researcher could start from different conditions in a different organisation or community and test the predictions of each theory under those different
conditions. The predictions of the theory would have to be matched to the new circumstances and would generate new predictions, but it would be the same theories that were being tested. So although the observations would be different, the case study's findings, that a particular theory was true, would be replicable.
Generalisability in cases
Every community case is unique and non-repeatable. The outcome of a single case study is therefore never going to be generalisable against any other unique case situation. However, this is no different from the s ituation in the natural sciences. Any one experiment is only true under the particular set of circumstances it was conducted in. It is only when other experimenters using similar circumstances have repeated the findings that the experiment can be said to be generalised. The
situation is no different with case research. The findings of one case can only be confirmed or refuted by reference to other similar cases with similar circumstances. Generalisability is a quality of theories which have been tested under different circumstances and applications, and until it has been so tested no theory is generalisable. The generalisability of case study research is therefore directly comparable with natural science research's generalisability.
In looking at the requirements for scientific rigour then, it can be seen that case study research is equally as valid as any other type of research carried out under the natural sciences model.
3.3 Action Research
3.3.1 Action Resea rch and the case approach
The choice of the action research methodology as the underpinning for the cases in this thesis follows logically from the underlying philosophy of community
informatics: people and their interactions are central to the process. Action research is a subset of case study research (Galliers 1 99 1 ). In the standard systems life cycle, integration of the computer system into the social system of the user community only occurs in the final stage of the methodology, if it is consciously done at all. In the socio-technical methodology that underlies community
informatics, the integration of technology with social factors and economic benefit is seen as the prime objective, and not an adjunct to the other factors. The socio technical philosophy is based on empowering the end users to become part of the design environment and to actively participate in all stages of the design cycle. However, this is the statement of an ideal: in practice not all community members are capable or even willing to participate, so design methods have to be chosen which, while complying with the spirit of socio-technical design, do not strictly adhere to the purist position. Action research has developed as the best way of
combining the ideals of participation with the ability to effect significant change in the situation.
Action research combines a specific mix of pure research and empirical processes. The research decision is made to study a particular organisation at a particular time in the search for situation specific knowledge. The researcher therefore deliberately chooses to sacrifice external validity for i ncreased internal validity, and increased relevance. The research then applies the i ntervention, and by empirical enquiry asks a generalised question to get a generalised answer that will lead to a more focus sed question. The cycle continues asking general questions in order to get to more refined questions, until underlying relationships begin to appear. It then possible to use an inductive process to begin to generate 'grounded theory'
hypotheses to explain the relationships being uncovered. The theory is then tested by new intervention, and refined according to whether the intervention was
successful or not. Without this iterative process the research becomes 'unevaluated, one-shot action' (Swepson 1 995) rather than action research.
Action research involves reliance on a theory in choosing the case situation, and planning the intervention. Action research provides a means of substantiating that theory and at the same time it provides material for developing the theory - which may in turn suggest more action. The action researcher is therefore engaged in a process of examining, hypothesising and effecting change in a community where there is the expectation of a contribution to knowledge, and also to produce knowledge that can be directly applied to improve the current situation. The research is sensitive to, and informed by, theoretical principles being used, so that these can be transcended, and new theory developed as the researcher combines the roles of researcher and developer (Gummesson 1 99 1 ).
3.3.2 Stages of the Action Research cycle
Action research is used to combine action with research, to span the gap between practice and theory. Action researchers participate in the research situation and interact with the phenomena they are studying in order to test an existing theory or to develop a new theory (Argyris and Schon 1 989). An action research project involves analysis of a problem situation not under the control of the researcher, the making of plans for an intervention in the situation, and the attempted execution of those plans (Mansell 1 99 1 ). Action research follows a standard model but the detail
is usually adapted to suit the precise circumstances of its use. The modem five step process emphasises the socio-technical systems perspective (Susman 1 983). The basic method is an iterative cycle of planning, action and reflection following a negotiated terms of reference. This sets out the responsibilities and roles of each of the participants and delineates the research environment. The terms of reference may be a written agreement, or more usually, an implicit understanding in how the research is conducted. Once the terms of reference are clear, a research cycle is followed progressing through the stages of diagnosis, action planning, action implementation, evaluation, and reflection.
Diagnosis
This step is concerned with identifying and defining the problem, the reason for the proposed change. This diagnosis involves analysis of the social and organisational structures from a holistic viewpoint. The outcome of this is theory generation, hypothesising reasons underlying the current situation. A crucial aspect of this stage is the role and status of the researcher. In the earliest forms of action research, the researcher acted as an expert, and worked directly only with the senior staff, and regarded the other staff almost as experimental subjects. However, this changed under the influence of the ideas inherent in soft systems methodologies (Checkland 1 999), and the socio-technical approach to systems development. The modem approach is one where the researcher, while still regarded as having specialist knowledge, does not assume a superior knowledge, and always works to achieve full participation of all staff.
Action Planning
This specifies the action to be taken to achieve the organisational aims. The actions are informed and directed by the theory arising from, and developed in, the
diagnosis stage. The planning stage establishes the action to take, the changes expected, defines the methods of measurement and the targets to be achieved.
Implementation
The exact nature of the interventio n varies with each case, but the essence is a close collaboration between the investigator and the participants within the organisation. The actions can be direct, addressed at the heart of the perceived problems, or indirect aimed at changing the environment to see what changes eventuate. The
implementation is usually carried out directly by the action researcher, with the cooperation of the community or organisation.
Evaluation
This stage looks at the situation as it emerges from the planned interventions. The effects may be what was expected or not, and may have changed the problems or not. If the change was successful, the researchers then evaluate whether the change was the result of their intervention, or had some other cause. Where the change was unsuccessful, or only partly successful, the theories need to be revised and
reconsidered, or re-examined for unexpected confounding factors. Reflection
This is actually continuous within the research cycle but is regarded as a separate stage in the model . After measuring the success or failure of an intervention, the researcher and the participants both need to consider the lessons learned from the