Design, method and conceptions of quality in policy research

Every research topic begins with a research objective (Blaikie, 2004), from which the design and method evolves and attempts an answer (Williams, 2003). This general objective and more specific objectives, unique to each individual piece of policy-based research, assist in formulating the particular approach of the study and the adoption of suitable methods and design (Becker & Bryman, 2004). The choice of research design and methodology used must also adhere to widely-held views on quality criteria with regards to social and policy research, and what is deemed ‘good practice’.

Becker and colleagues (2006), identify two contrasting approaches in assessing the quality of social research – measurement-led, ‘structured’

criterion, or value-led, ‘sensitised’ criterion - concluding that quality is construed in the ‘eye of the beholder’ and is largely affected by the degree of orientation to ‘research process’, ‘policy’, ‘theory’, ‘service user’ or ‘academic prestige’ drivers of investigation.

Becker et al. (2006) on the basis of a survey of over 250 social policy researchers suggest that the most important quality criteria in social research are:

 accessibility to the appropriate audience

 a research design clearly addressing the research question

 transparency of data collection and analysis

 an explicit account of the research process, design, method and analysis of data

 a contribution to knowledge, and

 adherence to issues of informed consent, safety and ethical codes and protocol.

These quality criteria are linked to notions of originality, significance and rigour – free from values. However, the comprehension of ‘originality’ ‘significance’

and ‘rigour’, as standards of quality in research is subject to a diversity of

opinion, with persisting problems of definition and interpretation (Becker et al., 2006). Becker and Bryman (2004) therefore suggest that it may be more worthwhile addressing the core process of policy research – the critical relationship between theory, analysis and procedure rather than to focus too much on originality, significance and rigour. Tackling this interface of theory and practice is more pertinent to the objectives of research (Becker & Bryman, 2004) and assessing its quality (Becker et al., 2006).

As Hart (1998) observes, research can generally be classified according to its design features and its intended outcomes. Also, a key element of good research is integration. Integration is about making connections between ideas, theories and experience. It is about placing some episode into a larger theoretical framework, thereby providing a new way of looking at that phenomenon (Hart, 1998).

As concerns rigour even the highest transparency of data collection and analysis and an explicit account of the research process, design, and method cannot ensure the neutrality of the research methodology and/or findings.

Research texts are not ‘neutral’ writings offering a ‘Gods eye’ view of the world;

value-free assessment is therefore an ‘impossibility’, with the very claim to being ‘value-free’, a value in itself (Dalby, 2003). Inclusion and exclusion of information reflect the personal and political motivations and bias of the author (Creswell, 2003). Much research exists in a supposed vacuum, quite apart from the vagaries, whims and real-life relations of everyday existence, but this does not mean it fails to detail, to colour in, contribute, or penetrate individual or institutional understanding, or reflect a perceived reality, or be itself affected by social and political values.

In conducting and disseminating policy research it is important to critically evaluate the implications of social and political aspirations and ideologies, and also the widespread commitment to the ideology and impact of evidence-based research amongst the public, politicians, practitioners, professionals and their increased collective involvement; this could enliven the process and ensure progress (Becker & Bryman, 2004). Moreover, closing the gap between research and practice would need practitioners to understand the relative strengths and weaknesses of different research methods and design and how they answer, or provide evidence on, different questions (Becker & Bryman,

2004). If this could be achieved, within the refreshed parameters of theory and practice, a space for the discussion of ‘quality’ and ‘utility’ could be adequately located (Becker et al., 2006).

In brief, what counts as quality, much like what counts as reality, is open to significant ontological interpretation and the powers of perception and perspective (Becker et al., 2006). Quality in research is incapable of being universally applied, though, perhaps capable of being individually assessed and rationalised (Bryman, 2004).

An absolutist research-based ‘quality hierarchy’ of ‘universal utility’ – although somewhat perceived in the social sciences – is at the very least, irrelevant to the debate (Becker et al., 2006). Nonetheless, a consensus of judgement, in the quality of research is perhaps well-founded and accepted, particularly if a more specific and less general overview is adopted, with regards to approaching and adopting research design and method (Becker et al., 2006).

In this context, the primary aim of this research is to inform future occupational safety and health policy development by examining the impact of existing policies for managing psychosocial risks at work. At the same time this research seeks to raise awareness amongst policy-makers about the importance of psychosocial risks within the realm of all occupational hazards.

Ritchie and Spencer (1994) outlined the four categories that need to be addressed in applied policy research:

i) Consideration of the context and identification of the form and nature of what exists;

ii) Diagnostics to examine the reasons for, or causes of, what exists;

iii) Evaluation and appraisal of the effectiveness of what exists, particularly in terms of barriers of implementing the system; and

iv) Development of a strategy to define approaches to overcome barriers.

This research attempts to cover each of these four categories by using a mixed methods approach.