focused on determining pragmatic approaches to data analysis using the information available. Therefore, the objectives are to:
• Understand the performance of the standard binary and augmented binary methods in small sample sizes
• Determine the differences in performance when using generalised least squares (GLS) and generalised estimating equations (GEE) for modelling the continuous
component in the augmented binary method
• Identify the most appropriate way to express the treatment effect estimate based on performance characteristics (risk difference vs. odds ratio)
• Identify and implement appropriate small sample corrections and compare the performance with the uncorrected methods
• Determine the most efficient analysis methods to reduce the required sample size in a rare disease trial or to report the treatment effect more precisely
• Make analysis recommendations for trials in rare diseases using composite end- points
The chapter proceeds as follows. We introduce the methods and data that we will use to investigate the performance through re-sampling. We show the behaviour of the methods for varying sample size and response thresholds. We include a simulation study to verify the findings of the re-sampling and conclude with a discussion and recommendations for analysing rare disease trials using these endpoints.
2.2
Small Sample Adjustments
2.2.1
Binary Component Adjustment
Albert and Anderson show when fitting a logistic regression model to small samples that, although the likelihood converges, at least one parameter estimate may be theoretically infinite [45]. This phenomenon is commonly termed ‘perfect separation’ and occurs if the model can perfectly predict the response or if there are more parameters in the model than can be estimated because the data are sparse [46]. Firth provides an alternative to maximum likelihood estimation (MLE) in these circumstances [47].
This involves using penalised maximum likelihood (PML) to correct the mechanism producing the estimate, namely the score equation, rather than the estimate itself. More generally, penalised maximum likelihood can be thought of as a technique to introduce a small amount of bias in the parameter estimates in order to circumvent problems in the stability of parameter estimates that arise when the likelihood is relatively flat [48].
The penalised likelihood is shown below in equation (2.1), where L(θ) is the usual likelihood function for a logit model and I(θ) is the information matrix.
L∗(θ) = L(θ)|I(θ)|12 (2.1)
As maximum likelihood estimates are always biased away from zero for logisitic regression in small samples, bias correction therefore involves some degree of shrinkage of the estimate towards this point [47]. This results in the method also reducing the variance, so that bias reduction does not necessarily lead to a substantial loss in power. This is an important and attractive property of the Firth correction, as a trade-off between bias and variance usually exists.
We can intuitively understand why the correction results in variance reduction. In this setting, the corrected estimate is always closer to zero, therefore it must always have a reduced variance due to being bounded between zero and the uncorrected estimate. The source of the bias is curvature in the score function s(y, θ), meaning that if the score function is decreasing and curved in the area around the true parameter θtrue
then a high miss s(y, θ) > 0 implies an estimate well above the true value, so that
θmle >> θtrue and a low miss s(y, θ) < 0 implies an estimate only slightly below the
true value, so that θmle < θtrue. This implies that low misses and high misses do not
cancel and that the MLE is too large on average. This is discussed further in [49]. We will use the Firth correction in both the standard binary method and the multiple logistic regression models in the augmented binary method. The modified estimator can be easily implemented in R using the brglm package [50], which provides the penalised likelihood estimates.
2.2.2
Continuous Component Adjustment for GEE
For continuous longitudinal data, there are a number of estimators providing estimates
ˆ
θ for θ. The generalised least squares (GLS) estimator is typically used in linear
2.2 Small Sample Adjustments 25
estimator of θ. The validity of the inference from the GLS estimator is dependent on correctly specifying the subject mean µij and variance Vi. The mixed model allows both marginal and subject specific inference. Generalised estimating equations (GEE) allow only for marginal inference. However, valid inference is possible from ˆθ in the GEE
method if µij is correctly specified, even if Vi is misspecified [51]. We are interested in determining whether the estimator used for the continuous component has a substantial effect on model performance. In larger samples we may expect the differences between GLS and GEE to be negligible, however in small samples the estimation method may be more influential.
GEE is typically considered to be a more robust method for model misspecification, particularly as the variance estimator that is commonly used provides robust standard error estimates. This robustness property could be desirable in this setting, where model misspecification may be more problematic for the variance estimates than in a larger sample. However when using these methods where the number of patients is small, the robust standard error estimates are subject to downward bias leading to inflated type I errors [52]. The standard robust sandwich covariance estimator is shown in equation (2.2).
Vsand= (Pni=1DiVi−1Di)−1(Pni=1DiVi−1Cocv(Yi)Vi−1Di)(Pni=1DiVi−1Di)−1 (2.2)
where:
Di = ∂µ∂βi
µi is the vector of mean responses β is the parameter vector
Vi is the working variance-covariance matrix for Yi
Cocv(Yi) = (Yi−µˆi)(Yi−µˆi)′
To address the limitations of this estimator in small samples, Morel, Bokossa and Neerchal [53] propose a correction which inflates the variance estimate. The small sample adjusted variance estimator VM BN is shown below.
VM BN = Pn i=1DiVi−1Di −1 Pn i=1DiVi−1 kCovc(Yi)+δmξVi Vi−1Di Pn i=1DiVi−1Di −1
where:
k = N −1N −pn−1n
p is the number of parameters
N is the total number of observations n is the number of patients
δm = p n−p, if n > 3p 1 2, otherwise ξ= max 1, trace (Pn i=1DiV −1 i Di) −1 (Pn i=1DiV −1 i Cov(Yi)V −1 i Di) p
An appealing property of this estimator is that as the sample size increases, k → 1 and δm → 0, so that VM BN → V. Note in ξ that the sum of the eigenvalues
is used. These eignevalues may also be referred to as ‘generalised design effects’ [54]. Alternative corrections may include a different function of the eigenvalues, such as the maximum or the product, which corresponds to the determinant of Pn i=1DiVi−1Di −1 Pn i=1DiVi−1Cov(Yi)Vi−1Di
. However in this case we employ the trace, as demonstrated by Morel et al. [53]. We implement this variance correction in R using a modification of the code provided in the geesmv package [55] when using the GEE estimator for the continuous component.