Assessment of publication bias

The assessment of publication bias includes both a graphical method knowns as the funnel plot as well as analytical methods to test hypotheses regarding the presence or absence of “small study effect”, that is, the association between treatment effect and

105

sample size (Sterne et al. 2001, Song et al. 2013, Bland 2015). Much of the discussion around publication bias centres around RCTs of treatment effects and the impact of the unpublished studies resulting in negative findings on a meta-analysis of treatment effects (Egger 1997, Leandro 2005, Sterne et al. 2001). This does not apply to the synthesis of incidence rates observed in a population, as explored in this meta-analysis. Consultation with the University statistician regarding the possibility of exploring publication bias statistically took place following the update of the systematic review in 2016. It was decided that since the meta-analysis was dealing solely with incidence rates rather than risks or associations, that the statistical exploration of publication bias would not be required at this stage. This may be a step to be considered in future work undertaken.

4.3. Results

This section presents the outcomes of the latest version of the systematic review and meta-analysis, it therefore combines the findings of the 2010 and 2016 studies.

Where appropriate, reference is made to decisions taken in the 2010 review if these differ from the 2016 version. Two checklists guided the presentation of the results: The PRISMA statement (Moher et al. 2009) (appendix 4.5) and the MOOSE statement (Stroup et al. 2000) both of which were developed to encourage high quality of reporting for SRs and MAs (Greenhalgh 2015), with the MOOSE statement being more specific to the reporting of SR and MAs of observational studies in particular (Sanderson et al.

2007).

106 4.3.1 Study selection

28 studies met the inclusion criteria for this review. Figure 3 (p.109) uses the PRISMA flow chart (Moher et al. 2009) to summarise the study selection process that was followed as the selection criteria outlined above were applied. This flow chart consolidates the process of study selection from 2010 as well as 2016. Keyword combinations returned a total of 6623 citations, hand searching resulted in eight citations, once duplicates were removed, 6250 titles were screened for eligibility, 6023 rejected due to irrelevance. Following the application of the inclusion / exclusion criteria, 172 studies were rejected leaving, 55 full texts which were retrieved for scrutiny. All of these studies were read carefully, 27 were excluded due to various reasons: eight publications contained no new data: McManus et al. (2009b), McManus et al. (2011), van Rijsbergen et al. (2011), Oldenbeuving et al. (2013). See below for a description of decisions on duplicate publications; three systematic reviews: Carin-Levy et al. (2012), Shi et al.

(2012), Ojagbemi and Ffytche (2016); one study protocol by Klimiec et al. (2015). Four publications dealt with treatment approaches to delirium in acute stroke: Oldenbeuving et al. (2008), Ohta et al. (2013), Duan et al. (2016) and Rice et al. (2016). Other reasons for exclusion were language, single case studies, one paper studied delirium in the post-acute phase (Turco et al. 2013), one survey of practice (Carin-Levy et al. 2013) and one paper which did not distinguish between stroke and TIA and dealt with cognitive impairment generally rather than delirium per se (Pendlebury et al. 2011).

Several instances of duplicate publications (discussed in section 4.2.3.5) had emerged, these were dealt with carefully, case-by-case, details as follows:

In the original 2010 review, two publications by the same group emerged (McManus et al. 2009a and 2009b). Both papers described the identification of delirium in the same cohort of stroke patients, without cross-referencing the other study within the

107

methods section, a matter which is considered crucial when publishing multiple studies presenting data from the same cohort (Haworth et al. 2014). One paper, published in Age and Ageing (McManus et al. 2009a) described identification of delirium in a cohort

of 82 stroke patients using the CAM. The second paper, published in the International Journal of Geriatric Psychiatry (McManus et al. 2009b) compared the CAM and the DRS

on the same cohort of patients. At the time of the 2010 review, this matter was discussed within the supervisory team and a decision to include both papers was taken, based on the fact that there was a small amount of studies found and the incidence rates found for delirium using either tool were not identical. It was therefore decided that for the purpose of the meta-analysis, the incidence rates of delirium found using the CAM (28%) could be used as one set of data (McManus et al. 2009a), and the DRS rates (27%) could be used as a separate set of data (McManus et al. 2009b). This decision was later reversed when conducting the 2016 update, following careful consideration and realisation that including one cohort twice is likely to give added weight to the cohort (Wood 2008). The decision was therefore taken to include only McManus et al. (2009a) and exclude McManus et al. (2009b).

The 2010 review included a pilot study of a drug treatment which was nested in a large observational study (n=527) (Oldenbeuving et al. 2008). In the 2016 update, four other publications by the same group of authors, describing the same cohort of participants were identified (Oldenbeuving et al. 2011, van Rijsbergen et al. 2011, Oldenveuving at al. 2013, Oldenbeuving et al. 2014). Following careful scrutiny of these papers the following decisions were taken:

 Oldenbeuving et al. (2011) to replace Oldenbeuving et al. (2008) due to the data presented in the 2011 publication being more relevant to the SR question, and the

108

2008 publication being a pilot study of a drug to treat delirium in a cohort of stroke patients.

 Oldenbeuving et al. (2013) to be excluded as it presents data on the same cohort of n=527 already included in the SR (Oldenbeuving et al. 2011).

 van Rijsbergen et al. (2011) was to be excluded as it was a case-control nested within the data presented by the publication by Oldenbeuving et al. (2011), which was already included in the review.

 Oldenbeuving et al. (2014) presented a model of predicting delirium in two cohorts, the first of which was the cohort presented in the 2011 study, the second, a new cohort of 273 participants. The diagnostic procedures in this new cohort were clearly described and it was therefore possible to include this publication in the SR and MA.

109

Figure 3: Study Selection: PRISMA flow chart (Moher et al. 2009)

110