Chapter 7: Discussion and Conclusions
7.6 Sustaining implementation of evidence-based practices
This cohort study was implemented as one component of an ongoing scholarship of practice partnership. Randomised controlled trials and subsequent meta-analysis are responsible for generating much of the evidence of efficacy of IPS. RCTs and meta- analyses are recognised as the highest level of evidence for intervention (Howick 2011). Prognostic research, such as this study, where the aim is to predict outcome high-quality prospective cohort studies with adequate power, offers the best evidence (Hemingway et al. 2013; Riley et al. 2013; Steyerberg et al. 2013; Hingorani et al. 2013).
There are inherent problems in RCTs; these challenges have been recognised in IPS research. The efficacy of interventions established in clinical trials may not be sustained in implementation in clinical services without the monitoring involved in managing RCTs (Bergmark et al. 2018). RCT often result in small-scale implementation across several remote sites (Talbot et al. 2018). Funded research enlists the support of funders, external champions and decision-makers in addition to the finance to support implementation: this may not be sustained when the RCT concludes that growth and maintenance of services are challenging in the context of cost pressures on the NHS (Hutchinson et al. 2018). The enthusiasm and support around trial implementation can wane when the research is completed (Hegelstad et al. 2018). Put simply, the efficacy established in RCTs may not be translated into effectiveness in real world day-to-day practice (Westfall et al. 2007)
This phenomenon has also been recognised by Noel et al. (2017), who recommend the value of a continuing learning community to sustain and generalize implementation (Latimer 2017). Local scholarship of practice offers a potential forum for this ongoing learning.
The OECD (2015b) suggest that secondary analysis of data gathered for clinical purposes offers opportunities to improve the implementation of new initiatives and ongoing evaluation. To achieve this aim, the NHS must establish mechanisms to support the secondary use of health data to increase impact (Singh & Sittig 2016). Such mechanisms create opportunities for researchers to answer clinical questions in real clinical situations (Okun et al. 2013).
This study was one example of a priority of the NHS: making use of clinical data to improve quality and deliver good value care. (NHS England 2018)
.
To make the most of this potential growing field of secondary data analysis, research must ensure the reliability and validity of data (Raghupathi & Raghupathi 2014; Scott et al. 2017).7.6.1 Comparing samples of RCTs and secondary data analysis
The gender split of this study (61% male) was almost identical to the pooled sample from good fidelity RCTs (62% male). However, concern has been raised by Kirkbride et al. (2012), who recognise that psychosocial intervention studies for people with CMHP tend to be around 64% male, while the actual population is more evenly split. This study also shared a disproportionate number of married people when compared to a population with CMHP. Around 17% of participants were married; whereas in the pooled sample of IPS studies, 26% were married. Yet national surveys suggest only around 0.5% of people with complex mental health problems are married.
It may be that more males have the motivation to work on vocational goals, and that people who are in significant relationships enlist the support of their partner to support their aspirations. Regardless, good quality VRS services should be available to all people with CMHP. Therefore, VRS should make an effort to increase recruitment of female clients: those without naturally occurring social support networks.
Of greatest concern, however, is the low numbers of young people accessing VRS. This is not a problem unique to this study. Only 30 (15%) of cases in this sample were aged 25 or younger; the mean age was 37.40 (SD10.80). The mean age of participants of the eleven studies included in the meta-analysis ranged from 21.29 (SD 2.39) (Killackey et al. 2014) to 42.90 (SD11.50) (Bond et al 2015). The study with the youngest age range was focused on early intervention; and the next lowest mean age was 32 (SD8.9) (Waghorn 2014). Therefore, it appears that VRS are not reaching young people.
The Schizophrenia Commission has acknowledged the value of early intervention service in delivering IPS to support young people in obtaining educational opportunities: citing that for every person who does not gain a university degree, there is a lost net benefit of £197,000 to society; and completing a college qualification is
likely to result in a 12% wage gain (Schizophrenia Commission 2012). The integration of IPS into early intervention services for people with psychosis is also highly recommended by the Work Foundation (Bajorek et al. 2016). If educational and employment goals are not addressed early, it risks individuals becoming stuck in mental health services (Ásmundsdóttir 2004).
Despite the small numbers of young people, age was established as a significant univariate predictor. This confirms the work of Burke-Miller et al. (2006; 2012); Salkever et al. (2007); and Tsang et al. (2010). Early exclusion from education and employment risks long-term detrimental effects on health and wellbeing, and establishes an economic burden on society (Sveinsdottir et al. 2018). IPS research to date, despite recognising the value of education, is often measured only on employment outcomes.
Killackey et al. (2018) highlight the value of assisting young people to complete education, as this improves an individual’s future career potential. This is supported by the significant negative relationship between not completing school exams and outcomes in both this study and Hegelstad et al. (2018). Killackey et al. (2018) recommend that a more targeted approach to educational outcomes may be required: the IPS fidelity scale should include references to educational outcomes, and recent studies have suggested the employment of an IPS worker with expertise in working in the education sector (Killackey et al. 2018).
7.6.2 Effectiveness of IPS cohort study compared to IPS RCTs
It is also important to compare the outcomes of this study to those of good fidelity IPS RCTs. The likelihood ratio of obtaining employment in this study was similar or better to many IPS RCTs. Good fidelity IPS RCT studies have a range of mean time to first job of 2.2 months (SD2.53) (Wong et al. 2008) to 6.6 months (SD6.21) (Gold et al. 2006). The mean of this study was 11.96 (SD 12.18), a considerably longer timeframe. This extended time probably owes to realistic IPS intervention in practice, as opposed to artificial time constraints of RCTs (Mueser et al. 2016).