Implementation for improved information anchoring

We note that if Va,o≈ VDF,oi.e. the variance of the parameters in the de-facto imputation model

matches the variance of the parameters in the de-jure (MAR) imputation model, the approximation of the information anchored variance by Rubin’s variance estimator is sharpened. When this is the case the information anchoring (4.5) becomes,

0 ≈ Eh ˆBDJ i Eh ˆWDJ iO(n −2_{) . (4.8)}

In the baseline and single follow-up setting under CR Va,o≈ Vr is obtained when the number of

fully observed patients in the reference arm is close to no—the number of observed patients in the

active arm at time 2— with the assumption of equal variance by arm.

If the number of observed active patients (no) and reference arm patients (n) are not similar

in a trial with a single follow-up, to achieve improved information anchoring in practice, we can simply scale the variance of the parameters in the CR imputation distribution by n/no. With equal

variance-covariance of the data by arm, which we denote by A, Va,o= A/noand Vr= A/n. After

scaling Vrby n/nothe variance of the parameters in the CR imputation model will therefore match

the variance of the parameters in the de-jure (MAR) imputation model.

Alternatively prior to performing reference based imputation we can bootstrap the observed ref- erence data in order to obtain a sample of the required size from the reference distribution, which will result in Va,o ≈ Vr. Bootstrapping involves repeated random sampling, with replacement,

of observed data and enables the estimation of the sampling distribution of statistics of interest [22]. In the current context, repeated sampling of the observed reference data can be employed to estimate the distribution of the data in the reference arm. For a simple trial with single follow-up we can then draw a sample of noreference patients from the estimated reference distribution. The

new sample of reference patients can then be used to construct the reference based imputation model, rather than the observed reference group patients.

For a trial with baseline (completely observed) and a single follow-up as explored in Proposition 1 after estimating the reference distribution via bootstrapping, we can draw a sample of n reference patients observed at baseline, of which no are also observed at time 2 to match the composition

of the active group sample used in MAR imputation. The newly drawn sample can then be used to construct the reference based imputation model. This will also result in improved information preservation for the J2R and CIR procedures.

The variance of the parameters in LMCF imputation models will be closer to the variance of the parameters in de-jure (MAR) imputation models when the number of deviators is small, i.e. no is

close to n. In the previous chapter we have already seen how the information anchoring property of Rubin’s rules is more accurate under LMCF imputation when the number of deviators is smaller.

Simulation study for improved information anchoring

To demonstrate the implementation for improved information anchoring in the CR, J2R and CIR settings we present results from an extension of the simulation study in Section 3.3.

Methods

The methods used to generate the datasets are described in Subsection 3.3.1. Interest lies in the mean treatment group difference at time 2. In this extension we consider the setting where ∆ = 0.3, as observed in the asthma trial, with 10–70% missing data at time 2 in the active arm only. As detailed in Subsection 3.3.1, for each missingness setting, the treatment effect and its estimated variance were computed using the full data. The analysis model was a linear regression

distributions to compute the treatment effect and its variance had we observed the post-deviation data in the particular de-facto scenario.

We then altered the implementation of the reference based MI approach of Carpenter, Roger and Kenward [13]. First we drew an independent sample of noreference patients with baseline and time

2 data, and a further ndwith baseline data only from the known reference population to match the

profile of the observed cases in the active arm. This sample was then used to build the required imputation models for CR, J2R and CIR imputation following the algorithm in [13], as outlined in Section 2.1. Of course, this approach is only possible in this simulation study set-up where the reference population distribution is known. Following imputation the sample of reference patients drawn for imputation were discarded from analysis. The analysis model was fitted to each imputed dataset in turn and results summarised for inference in the usual way using Rubin’s rules.

Secondly we bootstrapped the observed reference group data, using 1000 simple random draws with replacement, to estimate the mean and variance of the data in the reference group. We then drew an independent sample of noreference patients with baseline and week 12 data, and a further

nd with baseline data only from the estimated reference distribution. The bootstrap attempts to

mimic, ‘going back to the population,’ which is not possible in practice. This second sample was then used to build the required imputation models for CR, J2R and CIR imputation following the algorithm in [13]. Following imputation the sample of reference patients drawn for imputation was again discarded from analysis. The analysis model was fitted to each imputed dataset in turn and results summarised for inference using Rubin’s rules.

The main outcomes of interest were Rubin’s MI variance estimate for the treatment effect under the alternative implementations, and the ideal information anchored variance estimate, computed as in Section 3.1. Estimates were averaged over the 1000 simulations in each scenario. The main focus was to show how a superior approximation of the information anchored variance can be obtained by Rubin’s MI variance estimator under the alternative implementations, in comparison to the standard reference based MI implementation (for which results were presented in Section 3.3.2).

Results

Figure 4.2 shows excellent information anchoring by Rubin’s variance estimator under CR for all missingness scenarios when an independent sample of reference group patients, chosen to match the profile of the observed active patients, is drawn from the known reference population to build the imputation models. This is as expected since Va,o ≈ VDF,o. The approximation of the

information anchored variance by Rubin’s variance corresponds with that seen for the standard CR MI implementation for up to 40–50% missing data in the active arm, but is notably improved for 50% or greater missingness. This is only possible via simulation.

However, Figure 4.2 also shows excellent information anchoring by Rubin’s variance estimator under CR for all missingness scenarios when an independent sample of reference group patients that matches the profile of the observed active patients is drawn via bootstrapping the observed data. This could be done in practice if desired.

Figures 4.3 and 4.4 show this also holds for J2R and CIR imputation. The improvement in the approximation for missingness amounts above 50% is not quite as perfect as under CR, but in both cases it is superior to the standard reference based MI implementation for larger amounts of

missingness.

Our focus is not on the long-run empirical sampling variance (over the 1000 simulations) of the MI estimator here. However it is interesting to see what affect these procedures have on it. We see that when we draw a sample of independent reference group patients from the population for imputation the behaviour of the long-run empirical sampling variance changes (Figures 4.2–4.4). As discussed in Subsection 2.2.2 when we conduct reference based MI using the observed reference group data we introduce a covariance between the mean of the deviating active patients and the observed reference group patients. A consequence of borrowing information between trial arms. When analysing the treatment group difference this covariance is taken away in the computation of the sampling variance. Because the sample of reference group patients used here in imputation is an independent draw from the known population, and not included in the analysis, the unwanted covariance is no longer incorporated.

However, the data from the reference patients that are drawn to build the imputation model are not included in the analysis. There is therefore a mismatch in the conditioning used in the analysis and the imputation. This explains why the long-run empirical sampling variance still does not match Rubin’s variance estimator, i.e. the information anchored variance we are targeting. This property is discussed by Reiter in [59].

In practice we would have to use the observed data to estimate the parameters of the reference distribution. When we bootstrap the observed data to estimate the reference distribution for imputation, the empirical long-run sampling variance of the MI estimate is also unsuitable. It is smaller than the variance estimate we would obtain if the off-treatment behaviour of the deviators was actually observed. Moreover, since the data used to build the imputation model are not included in the analysis (we analyse only the original randomised patients data) there is also a mismatch in the conditioning used in the analysis and the imputation [59]. This is therefore why we also see that the empirical long-run sampling variance does not correspond with Rubin’s variance estimate and the information anchored variance via the bootstrapping implementation.

0 .002 .004 .006 .008 .01 Variance 10% 20% 30% 40% 50% 60% 70%

Proportion of missing data (active) CR − (n, no) from population 0 .002 .004 .006 .008 .01 Variance 10% 20% 30% 40% 50% 60% 70%

Proportion of missing data (active) CR − (n, no) via bootstrap

Deviators observed Rubin’s Information anchored Sampling MI

Figure 4.2: CR MI with implementation for improved information anchoring. Rubin’s variance estimator vs. information anchored variance vs. variance where deviators observed (all averaged over 1000 simulations) vs. long-run sampling variance of the 1000 MI estimates. ∆ = 0.3 and

n=250 per arm. 0 .002 .004 .006 .008 .01 Variance 10% 20% 30% 40% 50% 60% 70%

Proportion of missing data (active) J2R − (n, no) from population 0 .002 .004 .006 .008 .01 Variance 10% 20% 30% 40% 50% 60% 70%

Proportion of missing data (active) J2R − (n, no) via bootstrap

Deviators observed Rubin’s Information anchored Sampling MI

Figure 4.3: J2R MI with implementation for improved information anchoring. Rubin’s variance estimator vs. information anchored variance vs. variance where deviators observed (all averaged over 1000 simulations) vs. long-run sampling variance of the 1000 MI estimates. ∆ = 0.3 and

0 .002 .004 .006 .008 .01 Variance 10% 20% 30% 40% 50% 60% 70%

Proportion of missing data (active) CIR − (n, no) from population

0 .002 .004 .006 .008 .01 Variance 10% 20% 30% 40% 50% 60% 70%

Proportion of missing data (active) CIR − (n, no) via bootstrap

Deviators observed Rubin’s Information anchored Sampling MI

Figure 4.4: CIR MI with implementation for improved information anchoring. Rubin’s variance estimator vs. information anchored variance vs. variance where deviators observed (all averaged over 1000 simulations) vs. long-run sampling variance of the 1000 MI estimates. ∆ = 0.3 and

n=250 per arm.

Discussion

Improved information anchoring can be achieved by Rubin’s variance estimator if reference based imputation is conducted using a sample of reference patients that has the same profile of the observed active arm patients. This is obtained via bootstrapping the observed reference data. However, little is gained over using the standard implementation for up to 40-50% missing data in the active arm, since as discussed in the previous chapter, Rubin’s variance estimator provides excellent information anchoring in these circumstances. A similar effect could be obtained by scaling the variance of the parameters in the de-facto imputation model.

Information anchoring will naturally be stronger for a trial with imputed reference based deviation data in an active arm when the number of observed patients in the active arm corresponds with the total number of reference patients. When the number of reference cases is considerably different to the number of observed active cases the proposed alternative implementation may be useful. However this is unlikely in practice.