Statistical analysis

Measurement of treatment effect

Meta-analysis was conducted with RevMan 5.

For binary outcomes, we calculated a pooled estimate of the treatment effect for each outcome using the pooled odds ratio (OR) and 95% CIs. If calculating a pooled OR was not appropriate, we calculated an estimate of the treatment effect for each outcome using the OR and 95% CIs.

For continuous outcomes, we calculated the mean change from baseline for each group or the mean post-intervention values and 95% CIs for each group. We produced a pooled estimate of treatment effect by calculating the mean difference (MD) and 95% CIs. For QoL, CFQ-R was the most frequently used questionnaire and so we calculated the MD and 95% CIs. No other questionnaire was used. For time-to-event outcomes, such as 'time to first pulmonary exacerbation', we used measures of survival analysis, and calculated hazard ratios (HR) and 95% CIs between different arms of the trial. Where included trials did not report change data, but instead presented absolute post-treatment data without baseline data so it was not possible to calculate change data, we planned to use absolute post-treatment data instead of change from baseline. However, if the report presented

67 baseline and post-treatment data for any outcome, we calculated SDs for the change from baseline, for example if the CI was available. If there was not enough information available to calculate the SDs for the changes, we planned to impute them from other trials in the review, where data were available and trials were similar (i.e. when they used the same measurement scale, had the same degree of measurement error and had the same time periods between baseline and final value measurement). If neither of these methods were possible, we planned to calculate a change-from- baseline SD, making use of an imputed correlation coefficient (methods described in section 16.1.3.2 in the Cochrane Handbook of Systematic Reviews of Interventions).165

Unit of analysis issues

Within these reviews, we only included results from RCTs of parallel design in which individual trial participants were randomised. We excluded cross-over trials, because they are not appropriate for evaluating therapies that potentially correct the underlying defect.

Heterogeneity

In systematic reviews, data from several trials are synthesised to create an overall pooled estimate. One can expect there to be differences in results between included trials. However it is important to determine whether this variability (heterogeneity) is due to either ‘random play of events’ (chance), in which case synthesis of data is appropriate, or other factors that further require investigation.183 Heterogeneity requiring further investigation can be categorised into three types: clinical, methodological and statistical. Clinical heterogeneity refers to differences in the participants, interventions and outcomes between each trial. When there is variability in study design and or risk of bias, it is known as methodological heterogeneity. Statistical heterogeneity considers both clinical and methodological heterogeneity and refers to differences in results being more different than expected if simply due to random chance.184 _{In this review, statistical heterogeneity is referred to} simply as heterogeneity.

We planned to assess heterogeneity through a visual examination of the forest plots.185 _{If the results} of studies ‘lined up’ on the forest plot, we assumed little heterogeneity. We also considered the I2 statistic together with chi2 _{values and their CIs.}184 _{This reflects the likelihood that the variation of} results across trials was due to heterogeneity rather than chance, and we interpreted the I2 _statistic using the following classification:

 0% to 40%: might not be important;

 30% to 60%: may represent moderate heterogeneity;

 50% to 90%: may represent substantial heterogeneity;

68 This test was designed for analysis of heterogeneity in meta-analysis including large number of trials. Therefore we interpreted these test results with caution when only a small number of included trails were included.184

Data synthesis

Meta-analysis can use either a fixed effect or a random effects statistical model. A fixed effect model is used when there is no heterogeneity between included studies so the effect estimate of the treatment is the same between including trials and any variability is due to chance alone. A random effects model assumes that estimates of treatment effect vary between included trials due to both real differences in treatment effect and chance.186 _{In the potentiators review, we used a fixed-effect} model to analyse data from trials that we did not consider to be heterogeneous. When substantial or considerable heterogeneity was present (I2_{greater than 50%), we used a random-effects model to} analyse data.

In the correctors review, different interventions were being assessed (clinical heterogeneity) and so we always employed random effects model.

Subgroup analysis and investigation of heterogeneity

We planned to investigate any heterogeneity that we identified using subgroup analyses of potential confounding factors, if sufficient numbers (at least 10 trials) were available. For this review, we planned that these confounding factors would be:

 age (children (defined as younger than 18 years of age) versus adults);

 gender;

 different mutation classes (Table 1)

As we did not seek individual patient data from trial investigators, we did not undertake a subgroup analysis on the basis of disease severity.

Sensitivity analysis

If we had been able to combine a sufficient number of trials (at least 10), we planned to examine the impact of risk of bias on the results examined by comparing meta-analyses including and excluding trials with concerns of high risk of selection or reporting bias due to issues relating to randomisation, allocation concealment, or masking of interventions from participants or trial personnel.

69

Chapter 3

Results

In this section, I have highlighted the main findings of the reviews that are attached to this thesis. I have also report the results of the discussions held between the Cochrane Cystic Fibrosis Systematic Reviews team whilst conducting the reviews. For a more detailed report of the results please refer to result section of the completed reviews, attached in the appendix.

3.1 Summary of findings from the CFTR potentiators review