The present study uses the success factors and methodological issues discussed in this chapter as a foundation for the approach taken and the research design (summarised in table 2 & table 3) to indicate how these factors have influenced the design of this study. These are evaluated in this thesis in relation to a recently developed process evaluation framework by Nielsen and Abildgaard (2013).
As awareness of the importance of process evaluation has grown, a number of frameworks have been advanced to provide guidance about the factors that should be considered (e.g. Nielsen & Randall, 2013) and how they might integrate with established stress-management
78
cycles (e.g. Nielsen & Abildgaard, 2013). While Nielsen and Randall (2013) provide a particularly comprehensive framework, it is recognised by those authors and Nielsen and Abildgaard (2013) that individual studies are unlikely to be able to cover all potential process- related factors, and therefore some focus is necessary. Nielsen and Abildgaard’s (2013) framework provides a pragmatic approach to process evaluation that closely parallels stepwise approaches to intervention design and implementation (e.g. Cox, Griffiths, & Rial-González, 2000; HSE, 2007; Kompier et al., 1998) by breaking the process and evaluation into discrete phases; 1) initiation, 2) screening, 3) action planning, 4) implementation, and 5) effect evaluation.
Nielsen and Abildgaard propose that each stage can be considered as the outcome of the one before and as such, they advocate that each stage is evaluated separately ‘to detect how the decisions made and actions taken at one phase influence subsequent phases’ (p.282). Nielsen and Abildgaard’s framework therefore acknowledges Cox, Karanika, Griffiths, and Houdmont (2007), who highlight the need to locate the causes of ‘failure’ so interventions can be modified, rather than unnecessarily discarded. Their framework also acknowledges the role of the influential factors discussed during this literature review, such as context, and employees/stakeholder ‘mental models’ (e.g. ‘readiness for change’, motivation, perceptions of intervention activity) that can influence behaviours towards the intervention. However, the principal feature of the framework of interest here is the concept of each phase being considered as an outcome of the previous phase.
In viewing the process and phases in this way, the present thesis builds up a ‘programme theory’; in other words, how the intervention is supposed to work (Biron, 2012). However,
79
within a complex and dynamic environment where the specific interventions themselves are to be developed as part of the intervention project this ‘programme theory’ can be applied to the project as a whole and provides a simple model of how the phases contribute to each other and the overall process. Therefore each phase could be said to have particular ‘goals’, and evaluating how effectively these were met should go some way to pinpointing more precisely where the process may have been faulty, or particularly effective. Consequently, this study focuses on one or two questions for each phase, while also incorporating some of the most widely raised factors drawn from the research described in this literature review.
The Nielsen and Abildgaard framework (figure 1) begins with the ‘initiation’ phase (chapter four of this thesis); this phase concerns the project background and preparation for the baseline assessment and influences the ‘reach’ and effectiveness of this assessment (the ‘screening’ phase; chapter five). The accuracy of findings from the baseline assessment then influence the next stages, where decisions are taken on how those findings are translated into plans and actions (‘action planning’ & ‘implementation’ phase; chapter six): more accurate baseline data should enable more effective tailoring of actions to the needs of the workforce. Logically, if interventions are appropriately targeted and implemented, this in turn is likely to manifest itself in the final phase - the ‘effect evaluation’ (chapter seven); i.e. assessing whether and how interventions were effective. This simplified outline of each phase is expanded on in each chapter, as are the main aims of each.
Figure 1 also indicates the role of ‘change mechanisms’ throughout the process; Nielsen and Abildgaard (2013) characterise these as the ‘organisational actors’ mental models and behaviours’. The latter refers to the decisions and actions of the ‘organisational actors’; i.e.
80
the stakeholders - employees, management, and the steering group overseeing the process. Meanwhile, the ‘mental models’ referred to by Nielsen and Abildgaard’s framework, in figure 1, is a broad heading that has been used to refer to a wide range of factors in the process evaluation literature (e.g. Havermans et al., 2016). Nielsen and Abildgaard (2013) are not prescriptive in asserting which of these process and contextual factors should be assessed at each phase, which Fridrich, Jenny, and Bauer (2015) suggest may make their framework too general and leave researchers too much scope for interpreting what should be measured, and where. Nielsen and Abildgaard recognise there are still questions in terms of exactly which process evaluation-related factors are most important. However, this literature review has identified several factors and issues implicated in previous research, so although the exact role of these factors cannot be known prior to the project they do represent a selection of the most commonly reported process and contextual issues from the literature. Consequently, these are used to focus the evaluation, alongside the five phases, but in line with the developing nature of this field of research, the study will also document other particularly important factors that emerge during the course of the project.
81
Figure 1: Outline process evaluation framework and thesis structure (figure adapted from Nielsen & Abildgaard, 2013; p.281)
Figure 1 illustrates the overall framework, and how it fits with the structure of the thesis, while table 4 summarises the main focus of each phase and chapter. The final chapter, the discussion, begins by evaluating the process as a whole, then draws together and discusses the main factors that influenced the process. The second half of the discussion chapter then discusses the research from a methodological perspective and opportunities for future research.
82
Table 4: Overview of thesis process evaluation structure
Phase Main aim: project Main focus: chapter/research
Initiation (chapter 4)
• Planning and implementation of baseline assessment.
• Detailing the context; initiation and motivation for the project
• The ‘reach’ of the baseline assessment (e.g. survey response rate)
Screening (chapter 5)
• Identifying the priority stress-risk factors to assist with planning and implementation of intervention activity.
• The findings from the baseline stress-risk assessment
• Describing how key findings were arrived at and their accuracy
Action planning &
Implementation (chapter 6)
• Translation of findings from the previous phase into intervention plans
• The extent to which interventions addressed the needs identified by the baseline assessment
• How effectively interventions were implemented – specifically, the ‘reach’ and perceived quality of interventions
• Organisational events occurring during the process • Implementing planned intervention activities Effect evaluation (chapter 7)
• Evaluating the outcome of the intervention activity
• Analysis of changes from baseline
• The effect of employees exposure to interventions and their perceived quality
• Assessing relationships between employees’ baseline levels of cynicism about organisational changes and
employees’ engagement with intervention activity