Reflecting on research methods

With PD's strong ethical grounding (Robertson & Wagner 2012), emphasis on co-ownership and yet practical focus, research methods used to draw conclusions from PD need to be consistent with its design methods and approaches. I have identified five characteristics of PD that need to be shared or addressed by research methods if they are to be usable with my research.

•

Linked to practice and the real world

In the words of Bruce Archer, practice-based research is executed “through the medium” of practical action (Archer 1995). The kinds of issues being addressed in my project, in common with any PD project, are complex, have no singular definition and change in their constant interaction with real-world events. They are complex problems which have no right or wrong solution. The practical output therefore is not proposed as a singular, correct, solution, but a collaborative exploration and experiment with potential solutions. A research methodology therefore, must be able to reflect and value the complexities of practice and the unstable, real- world context; able to document a unique, long-term engagement and respond positively to emergent themes.

•

Can account for researcher intervention and change

This research comprises an intervention that produces change, in contrast with research that positions the researcher as objective and the situation uncontaminated by the research. Therefore research methods must be able to take into the account the ethical and ideological position of the researcher and, to counterbalance this, generate research data that is reviewable by other researchers.

•

Situated

PD emphases its situated nature. Every situation is unique and action can be taken only once, so there is little scope for comparative studies or experiments in a scientific sense. Research methods must be able to take on board the uniqueness of a project but still be able to generate insights with broader applicability that contribute to knowledge.

•

Ethical

All research must conform to ethical guidelines. However, PD as a design method places strong additional emphasis on ethics. Research methods should be conversant with the ethics of PD which include values around inclusion, a concern for power relations and accountability to participants and the local situation. Research methods must be suitable for settings in which with the Researcher does not exert full control.

•

Provides knowledge relevant to practice

This is a practice-based piece of research, motivated by issues observed in practice. My motivation is to feed back into practice and so my research method need to produce results that are accessible and interesting to practitioners and have scope and claims applicable to practice.

Given the above, research methodologies to be considered are Design Research, Action Research and the Case Study, which are briefly surveyed here. These methods do not dictate any particular method of data collection and analysis, but they do constitute a set of

approaches and guidelines within which different forms of evidence can be presented and claims of validity made. I suggest that each of the surveyed methods are relevant and useful to this research, but I finally propose that they should be combined.

4.3.1

Design Research

Design Research is a set of methods and arguments looking at the ways that the practices and outputs of designers can play a part in knowledge production beyond the singular designed

artefact. There are multiple approaches to Design Research with several notable contributors. These range from design science perspectives that investigate the use of empirical methods within design practice to points of view that emphasise the intuitive knowledge and creativity brought to design (Bayazit 2004).

The activity of Reflection is seen by several writers as a key component of design research. Donald Schon - an academic and professional in Urban Studies - suggests the persona of the 'reflective practitioner' and the activity of 'reflection-in-action' as the distinguishing features of design research and by extension, design education. In Educating the Reflective Practitioner he illustrates, via case studies, the value of group work and dialogue in problem solving. For Schon, design activity is neither the testable, scientifically based problem solving activity of 'technical rationality' . Rather it is a 'constructivist' activity that creates something new from a unique context: 'a practitioner's feel for materials, on-the-spot judgements, and improvisations – the forms of his or her reflection in action – are essential to professional competence' (Schön 1983).

Coming from a PD perspective, Simonsen et. al (2010) see Design Research as a bridge between the two separate cultures of Art and Science and even, by extension, part of a move to recognise hitherto hidden or ignored features of scientific research that rely on intuition, tacit knowledge on the part of researchers, and less goal-orientated practical explorations of a problem spaces . These authors link this constructivist definition of design to PD, defining PD as one of several approaches to design that “constructively appreciate the systemic

interdependencies between designed objects, use and context, and between designers and the people they are designing for (Simonsen et al. 2010 p.6). In this formulation, case studies and reflective accounts are the medium through which Design Research is investigated.

Steven Scrivener has gone further in embedding reflection within design research concretely as a method – encouraging transparent reflection within creative as well as primarily problem- solving projects. He defines stages within a design project where reflection should take place and makes a distinction between 'reflection-in-action' (that takes place during defined design stages) and 'reflection-on-action' (that takes place after design stages or in space in between). For creative projects, Scrivener encourages the systematic documentation of these reflections to distinguish design research from straightforward production. In problem-solving projects, the practitioners approach to problems should be shown to be 'self conscious' and 'systematic' (Scrivener 2000)

Bruce Archer takes a rigorous and more wide ranging view of research in design. For Archer, design activity on its own, even if accompanied by the circulation of the design outputs or accompanied by background research, is not enough to comprise Design Research. In order to be research, practitioner processes and outputs much be directed towards a question and should be based on transparent sources of data and evidence. Archer proposes three different

configurations for practice and research: studies about practice, studies for the purposes of contributing to practitioner activities, and studies through the medium of practice (Archer 1995).

4.3.2

Action Research

In Action Research (AR), as in design research, processes of reflections are key. AR is a method of research based on repeated cycles of analysis, action and reflection. AR has been developed specifically to deal with situations in which the research itself generates change and, most importantly, offers a research methodology that takes into account multiple participants with different interests and contexts taking an active role. AR acknowledges the active role of the researcher; rather than striving towards objective observations and uses reflection as a shared mechanism to analyse past actions and plan future interventions. AR is collaborative and situated, in that interventions are unique to the context and people involved with them. Accountability to participants and their local concerns are key; research should be transparent and visible, and presented in ways that is accessible and appealing to participants and their communities. In some variants of AR, for instance Participatory Action Research, it is strongly held that participants should share the setting of direction and focus of research together with interventions. Most importantly, AR has been developed specifically to deal with situations in which the research itself generates change and, most importantly, offers a research methodology that takes into account multiple participants with different interests taking active roles

(Coghlan & Brannick 2014; Robson 2011).

Being designed for situations of intervention and change, AR is also able to cope with a situation where the research question may, at first, be “fuzzy” (Coghlan & Brannick 2014). Checkland and Holwell point out that AR does not need a strictly defined hypothesis at the outset, as required by the scientific method. Instead, 'research themes' may be identified, which allows research to become more tightly defined in response to practice and reflection

(Checkland & Holwell 1998).

Some researchers position AR as a highly self-conscious method in which reflection is used to provide insight into how researchers' experiences and assumptions shape their actions. Brannick and Coghlan suggest that a key method of Action Research is journalling and this should provide a way to record and develop reflective skills. The journal provides a record of events and a series of self-evaluations. Those self-evaluations should be used by researchers to reveal how their interpretations of events or 'inferences' might have led to decisions (Coghlan & Brannick 2014). Other approaches suggest gathering a broader spectrum of evidence, for instance using interviews and ethnographies (Berg & Lune 2004), as would be done in any qualitative research project. These methods can be used to stimulate further reflection as well as document decisions and actions.

4.3.3

Case Study Research

Case study research allows for work that is a deep description and exploration of a real-world situation or situations. The case study can combine multiple sources of evidence and data to

reach an understanding of a single situation. To go beyond simple description and constitute research, a case study should find a way to increase understanding of a bigger problem. While a case study may not, on its own, be able to prove or falsify a hypothesis, a case study should contribute to some larger grouping of studies that together shed light on a topic of shared concern (Berg & Lune 2004).

Yin (2013) suggests that three conditions should be taken into account when choosing a research method. Firstly: the "type of research question", and secondly the "extent of control an investigator has over actual behavioural events" and lastly whether the object of research is in the present of the past. He suggests that the Case Study is suitable for questions asking "how" or "why" – so this clearly fits with my question. As in my research, the Case Study generally focusses on events in the present. Most importantly however, Yin suggests that a Case Study does not require the researcher to have full control over every aspect of the research situation or activity, and the research should be adaptable yet still rigorous. This lack of researcher control and the need for adaptability is a key characteristic of PD.

In case study research, through combining different forms of data collection and evidence, different voices are heard – one of these can be the researcher themselves (Yin 2013). It is therefore suited to projects where the researcher is an agent of change. Because of these multiple forms of evidence, triangulation – or checking – of the inferences of that data are integral to the method and can be achieved in a number of different ways: via the data, by using different methods, by involving other researchers or by employing different theories (Simons 2009).

A practical information systems project such as mine generates a large array of artefacts, observations and documents. These range from documentation of design meetings to working artefacts such as emails and task tickets, through to technical artefacts such as usage logs. In addition, the project also seeks reflections from participants and the researcher. A case study is able to include all of these as potential sources and should be able to use them to contribute to its validity (Simons 2009)