Statistical framework

Two conceptual approaches

Meta-analysis consists in pooling risk estimates from individual studies by computing a weighted average of effect estimates, using the inverse of esti- mates’ variance as weights. This allows to give more weight to the more pre- cise studies, with view to maximise the precision of the pooled effect estimate. Mathematically, ¯ θ = Pk i=1wiθi Pk i=1wi with wi = 1 vi (6.1) with ¯θ being the pooled estimate and θi and vi representing study-specific

mean effect and variance.

The variance of the pooled estimate is the reciprocal of the sum of study- specific weights wi, i.e.

V ar(θ) = 1 Pk

i=1wi

(6.2) There are two conceptually different approaches to performing a meta- analysis known as the fixed-effect model (FE) or the random-effect model (RE). The FE model assumes that each study generates an estimate of a com- mon true treatment / intervention effect, subject to sampling error known as within-study variation. By contrast, the RE model recognises that studies are heterogeneous in some respect (e.g. they were drawn from populations that differed from each other), with such differences having an impact on their es- timated treatment effect (Borenstein et al., 2009). As a result, each study is assumed to provide a study-specific treatment effect. This introduces another source of sampling error in the pooled risk estimate known as between-studies

variance (τ2_{). Whilst study-specific effect estimates are not identical, they are}

assumed to come from a common distribution - typically taken as the normal distribution - that is centred at the pooled estimate.

The choice of conceptual approach to pooling each risk estimate has strong implications on the computation of the pooled estimate. Under the FE model, the pooled estimate is assumed to provide information about the best estimate of effect. Consequently, studies with the greatest precision, i.e. with the lowest within-study variance, will be attributed a much higher weight than the least accurate studies and will therefore strongly influence the value of the pooled estimate. By contrast, under the RE approach, the pooled estimate represents the average intervention effect across different study populations (Borenstein et al., 2009). This shifts the focus of interest from the estimation of a common effect to the characterisation of the distribution of effects across studies.

Computation-wise, although effect estimates are weighted by the inverse of their variance in both models, under the RE-model the variance includes within-studies and between-studies variance (Sutton et al., 2000). Denoting σi

within-study variance, we obtain the following expressions for study-specific variance:

vi = σi under the FE model and vi = σi+ τ2 under the RE model (6.3)

Replacing for vi, we obtain the following expressions for study-specific weights:

wi =

1 σi

under the FE model and wi =

1 (σi+ τ2)

under the RE model (6.4) The inclusion of between-study variance in the RE model has three con- sequences. First, the weights assigned to each study will tend to be more balanced under the RE model than the FE model. Second, since the variance of the pooled estimate is the reciprocate of the sum of study weights (see equa- tion (2)), in the presence of between-studies variation, the confidence interval of the RE-pooled estimate will be larger than the confidence interval around

the FE-pooled estimate. Third, in line with equations 6.1 to 6.4, if τ2 _equals

to zero, FE and RE models will yield identical results.

Choice of random effect as conceptual model

The studies identified by the present systematic review and those selected from Chen et al. (2008)’s review were chosen for their similarities in terms of study population, intervention, health outcomes and study design. Neverthe- less, as described in section 2.3., studies exhibit some level of heterogeneity especially with regards their methodology. Therefore, whilst studies are con- sidered similar enough for the pooling of their effect estimates to be pertinent, their heterogeneity should be taken into account to avoid pooled estimates and their confidence intervals to be misleading. The RE model was therefore preferred.

Interpretation of random effect estimation results in decision modelling

Whilst the random effect model is advocated to incorporate heterogeneity between studies, the use of random effect meta-analysis results to populate cost-effectiveness decision models, is open to a number of possible interpreta- tions about the source of heterogeneity between studies and how the target setting of the intervention under assessment may potentially differ from the ones in the studies included in the meta-analysis. (Welton et al., 2015). The expression of “target setting” presently refers to population characteristics, intervention definition etc.

Typically, the mean of the random effect distribution is interpreted as the true effect to be observed in the future. This assumes that the decision target setting is equal to the average setting of the included studies and that the pooled estimate is an estimate of the true underlying intervention effect (D) that has been observed under noisy conditions resulting from random measure- ment errors (with the bias across studies being centred on zero) (Ades et al.,

2005; Welton et al., 2015).

Although this common approach was followed when parameterising the model developed in Chapter 4, it is worth noting that alternative methods have been suggested to characterise decision uncertainty stemming from vari- ation in intervention effect. Importantly, this source of decision uncertainty should be distinguished from parameter uncertainty as it cannot be reduced by further information.

In the case where the target setting for the decision is assumed to be similar to the ones in the studies included in the meta-analysis. Ades et al. (2005), suggested to rely on the predictive distribution of the intervention effect in a new study. Whilst the predictive distribution will be centred on the mean of the random effect distribution (i.e. RE pooled estimate), its variance will be larger as it accounts for uncertainty in parameters (D and τ2_{), as well as in}

study setting (Welton et al., 2015), namely:

dpredicted ∼ N (D, τ2) (6.5)

with D being the true underlying effect and τ2 _{the between-study variance in}

treatment effect.

Alternatively, if the decision target setting is expected to be made up of all the various target settings of the different studies included in the meta-analysis, it may be argued that there is not a single effect size but a distribution of effect sizes. In this case, quantification of the net benefit of intervention would ideally require to take the expectation of net benefit over the entire random effect distribution of intervention effect (Ades et al., 2005; Welton et al., 2015).