Review method

The review followed the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach.145 GRADE is recommended as it offers a transparent and structured approach for developing and summarising evidence for systematic reviews. GRADE is now widely endorsed by peer review journals and organisations such as the Cochrane collaborative and NICE.

Within GRADE, the scope of the review is firstly framed using the PICOS (Participants, Intervention, Comparison, Outcome, Study design) approach. Outcomes are then identified and rated in terms of their importance to patients.146 Evidence is systematically searched and assessed against the PICOS criteria. The quality of evidence for each patient-important outcome is then assessed across all studies, rather than for each individual study. A body of evidence is rated as high, moderate, low or very low quality (Table 2.3).

Table 2.3 GRADE quality definitions Quality level Icon Definition High

⨁⨁⨁⨁

We are very confident that the true effect lies close to that of the estimate of effect

Moderate

⨁⨁⨁_◯

MODERATE

We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of effect, but there is a possibility that it is substantially different

Low

⨁⨁_◯◯

LOW

Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of effect

Very low

⨁_◯◯◯

VERY LOW

We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect

Reproduced from GRADE147

Quality of evidence undergoes systematic ranking within GRADE. Figure 2.1 summarises the factors which can increase or decrease the quality assessment. Randomised controlled trials start as high quality evidence, and observational studies as low level. This is seen in the first and second columns on the left of Figure 2.1. Five factors can lead to the evidence being downgraded. This is shown in column three. Risk of bias across the studies, inconsistency between and across the studies findings, indirectness (indicated by differences in outcome measures, study populations, interventions or indirect comparisons) and imprecision of the findings (indicated by wide confidence intervals or small sample size) and publication bias can result in the initial quality assessment being decreased by one or two levels dependent on the severity. Three factors may result in evidence being upgraded. Where the studies demonstrate a large or very large magnitude effect, the initial quality assessment can be upgraded by one or two levels. The presence of a dose-response gradient may also result in evidence being upgraded, but by a maximum of one level. On occasions, all plausible residual confounding from

observational studies may be working to reduce the demonstrated effect or increase the effect, if no effect was observed. This can lead to evidence being upgraded by one level. Once all the adjustments to downgrade or upgrade the evidence have been made, the final quality assessment, seen in the far right column, is determined.

Figure 2.1 Summary of the GRADE approach to quality assessment

The table represents a summary of the factors affecting the quality rating.

The initial quality assessment is determined by the study design, however the quality rating can be downgraded and/or upgraded according to the above factors. The final quality assessment is shown in the far right column.

Adapted from GRADE147

When presenting results, GRADE recommends evidence profiles. The profile presents a detailed assessment of the quality of the evidence together with a summary of the findings for each important patient outcome. Evidence for diagnostic tests are often indirect, with research focusing on the accuracy of the diagnostic test itself rather than the effect of the test on patient important outcomes. Inferences must be made about the likely impact of the test in terms of the available outcomes (true positives, true negatives, false positives, false negatives). However, diagnostic accuracy can still provide important information to clinicians.145 In this situation,

cross-sectional and cohort studies start as high quality evidence but can move to low or very low quality dependent on other factors.147

2.2.3.1 The scope of the review

The scope of the review was outlined by the PICOS criteria as seen in Table 2.4.

Table 2.4 PICOS criteria for the systematic review

Participants Hospitalised children (0-18 years) on paediatric wards excluding critical care, theatre, accident and emergency

Intervention Development, use or evaluation of a PTTS to detect clinical deterioration

Comparison PTTS use (either alone or as part of a package of care) versus normal care

Outcome Any patient related outcome including (but not restricted to) death, admission to intensive care, cardiac and/or respiratory arrest, patient/family satisfaction

Any user related outcomes including (but not restricted to) staff satisfaction, inter-user reliability, utility

Study design All designs excluding reviews

2.2.3.2 Outcomes of interest

Primary outcomes were identified from the previous systematic review, the candidate’s knowledge of the literature and clinical expertise. The identified outcomes were then ranked in terms of their importance to patients using the framework from adult studies and the researcher’s clinical expertise (Table 2.5). No outcomes were ranked as being of low importance (rank 1-3).

Table 2.5 Patient important outcomes

Importance and rank Direct outcomes Surrogate outcomes Critical for decision making 9 Death 8 Cardiac arrest Respiratory arrest

CPR (chest compressions and/or bag- valve-mask)

Call for immediate assistance Code Blue

7 PICU admission

Severity of illness scores (PiM2)

Severity of illness markers (pH, lactate) Treatment markers (days of ventilation, length of PICU stay)

Important, but not critical for decision making

6 HDU

admission

Severity of illness scores (PiM2)

Severity of illness markers (pH, lactate) Treatment markers (days of non-invasive ventilation, length of HDU stay)

5 Length of hospital stay

Rapid response team activation

Urgent call to a healthcare professional 4 Low importance for decision making 3 2 1

Direct and surrogate outcomes were identified from the literature and candidates expertise. No outcomes were identified that were considered to be of low importance (rank 1-3).

Abbreviations:CPR: Cardiopulmonary resuscitation; HDU: High Dependency Unit; PiM2:

2.2.3.3 Inclusion and exclusion criteria

Papers were included if they described the development, testing or use of a PTTS to detect critical deterioration in children on hospital wards. Reports and reviews were excluded. Studies set in the emergency department, operating theatre or critical care unit were excluded as were those concerning both adult and paediatric patients unless the data relating to children could be adequately separated.

2.2.3.4 Search strategy

The following databases were searched: AMED, CINAHL, Cochrane Library, EMBASE, and OVID PubMed. A broad search strategy was adopted, informed by a previous systematic review of paediatric alert criteria,4 with Medical Subject Headings (MeSH) and free text searching using keywords in the title or abstract. Results were limited to papers from 1990 relating to children. Google scholar was searched using the terms paediatric early warning system/score, paediatric track and trigger system/score and paediatric rapid response/medical emergency team. Abstracts from the annual conferences of the Royal College of Paediatrics and Child Health (RCPCH), European Society of Paediatric and Neonatal Intensive Care (ESPNIC) and European Society of Intensive Care Medicine (ESCIM), together with the bi-annual World Congress in Paediatric Intensive Care were hand-searched from 2000 onwards.

After removal of duplicates, the title and abstract of records were screened by the candidate and the primary supervisor. Full-text papers were reviewed and the references and citations of eligible studies were manually searched on the Web of Science database. Uncertainty regarding inclusion of a paper was resolved through discussion within the supervisory team.

2.2.3.5 Data extraction:

Based on findings from the initial systematic review, three data extraction forms were developed for randomised control trials, observational studies and studies of diagnostic accuracy.4 The main categories of data are summarised in Figure 2.2. Extracted data were entered into Microsoft Excel for Mac 2011 (version 14.4.7).

Figure 2.2 Data extracted

Figure summarises the main categories of data extracted from papers identified for inclusion in the systematic review.

Abbreviations: AUROC: Area under the reciver operator characteristic curve; NPV:

Negative predictive value; PPV: Positive predictive value

2.2.3.6 Evidence appraisal and analysis.

PTTS were firstly categorised as ‘scoring’ or ‘trigger’ systems. Systems were then classified as being either ‘age-dependent’ (multiple systems with differing age- related thresholds) or ‘age-independent’ (a single system applied regardless of age).

Risk of bias for diagnostic accuracy studies were assessed using QUADAS 2148 (Appendix 5.1). Remaining quantitative studies were assessed against criteria in the GRADE handbook147 (Appendix 5.1). The risk of bias in qualitative studies was assessed using the Critical Appraisal Skills Programme (CASP) checklist (Appendix 5.3).149 Pooled risk ratio and 95% confidence intervals for each outcome were

Study Aim Location Setting Participants Methodology Method Participants Outcomes assessed Method of analysis Diagnostic accuracy Sensitivity Specificity PPV NPV AUROC Calibration System characteristics Trigger or score Age- dependant/ independent Age categories System parameters Number Simple or complex Classifi- cation Vital signs thresholds

calculated using GRADE Pro GDT150 and Vasser Stats.151 The overall quality of evidence for each patient-important outcome was ranked following the GRADE approach. Evidence profiles were formulated in GRADE Pro GDT.150

Where sufficient detail was provided, the risk ratio and 95% confidence intervals for each outcome were calculated. Results were separated into studies examining the introduction of a PTTS alone and those introducing a PTTS as part of a package of interventions, such as a rapid response team. Predictive validity was also summarised.