4.5 Handling Missing Data
4.5.3 Full Conditional Specification (FCS)
In contrast to specifying a joint model for the data, multiple imputation by fully conditional specification (FCS) (Buuren, 2012) adopts pre-specified conditional distributions for each variable in a data set. Each of these variables is then im- puted sequentially. When specifying the full conditional distributions for each variable, the data model must be defined, according to the type of data, and is taken to be conditional upon the other variables in the data set.
One issue with FCS is compatibility (Buuren, 2012). Another issue with FCS is convergence. Buuren and Groothuis-Oudshoorn (2011) suggested that con- vergence is achieved when the following two criteria are achieved: (1) different chains (i.e. posterior chains) of imputation are freely and stably mingled with each other and (2) the ’between-imputation’ variance between different chains is not larger than the average within-imputation variance.
Both the fully Bayesian framework and the multiple imputation by chained equa- tions are sampled from full conditional distributions. The fully Bayesian frame- work is a one-stage modelling approach, since the imputation model and the
substantive model are integrated, whereas the multiple imputation by chained equations is a two-stage modelling approach, since the imputation model and the substantive model are separately conducted in two stages. The fully Bayesian framework is described in Section 4.5.3.1, whereas the multiple imputation by chained equations (Stage 1) and response modelling (Stage 2) are described in Section 4.5.3.2 and 4.5.3.3 respectively.
4.5.3.1 Fully Bayesian Framework
Both Spiegelhalter (2003) and Lunn et al. (2006) discussed the fully Bayesian framework. The basic idea of a fully Bayesian framework for missing data in- volves specifying priors for all parameters concerned, along with specifying distributions for the missing data. These missing data are sampled from their conditional posterior distribution through a Gibbs sampler (Geman and Geman (1984); Spiegelhalter (2003)).
A fully Bayesian framework is an approach that not only imputes the miss- ing data, but also models the uncertainty of the imputation. It is a one-stage method, since missing data and parameters are imputed within the same sta- tistical model. In a fully Bayesian framework, each parameter is given a prior and all the parameters are initialised by drawing initial values from their corre- sponding priors. These parameters are then updated from their corresponding posterior functions, while also sampling the missing data from the likelihood of the imputation model, until convergence. In the case that there is missing data in the covariates, then a prior must be placed on these covariates.
The conditional prior for the covariate X, which is based on the parameter of covariates of the imputation modelψ, is denoted byp(X|ψ). After observing
the response data, Y, the prior is updated from a posterior. The probability density function for the response data Y, depending on the parameter Θ, is
CHAPTER 4. MISSING DATA THEORY, METHODOLOGY AND APPLICATION102 called likelihood function (p(Y |X,Θ)), which can be used to impute the missing
data. When there is missingness in the covariates, a prior distribution must be specified for the covariates as a part of the imputation model. In this case, the prior for the covariates will becomep(ψ).
In the fully Bayesian framework, the joint probability model for Y, X, M, Θ,
ψ,φ is as follows:
p(Y,X,M,Θ,φ,ψ)∝ p(Y |X,Θ)p(X |ψ)p(M|Y,X,φ)p(Θ)p(ψ)p(φ), (4.7)
(Ibrahim et al. (2005);Best and Mason (2012)) wherep(Y |X,Θ)is the likelihood of the substantive model, p(X|ψ)represents
the likelihood of the imputation model and p(M|Y,X,φ)represents the miss-
ingness model, p(Θ), p(φ)and p(ψ)are priors forΘ,φ andψ, respectively.
In this section, we focus on the case when the assumption of ignorability can be made. The non-ignorable case has been explained in Ibrahim et al. (2005). In other words, both φ and M terms can be ignored. The subsequent joint
probability model forY,X,Θ,ψ is therefore represented as follows:
p(Y,X,Θ,ψ)∝p(Y |X,Θ)p(X|ψ)p(Θ)p(ψ). (4.8)
A fully Bayesian framework utilises various samplers, including the Gibbs Sampler (Geman and Geman, 1984) and Slice Sampler (Ntzoufras, 2009) to up- date the parameters and missing values. This method begins with drawing the initial values forψ and Θparameters, as well as missing values ofX andY, by specifying the distribution terms of the missing values of X and the priors of
ψ and Θ. The missing values Xmiss, Ymiss, ψ and Θ are all sampled from their
sampling by the Gibbs Sampler is provided as Algorithm 4.1 below: Algorithm 4.1Fully Bayesian Framework
1: Specify the imputation modelX, p(X |ψ)and the likelihood function, p(Y |
X,Θ).
2: Specify the prior distributions for parameters of the substantive model and the imputation model,Θandψ respectively.
3: Initialiseψ,Θ,Xmiss andYmissasψ(0),Θ(0),X
(0)
missandY
(0)
miss.
4: fort=1, . . . ,T do
5: Sampleψ, Θ, X andY from the joint distribution function, p(Y,X,Θ,ψ),
to the following conditional posteriors:
a: Θ(t) from p(Y(t−1)|X(t−1),Θ(t))p(Θ(t))- to sample from this distribution,
proposeq(Θ(t))and accept according to Metropolis-Hastings ratio;
b: ψ(t)fromp(X(t−1)|ψ(t))p(ψ(t))- to sample from this distribution, propose
q(ψ(t))and accept according to Metropolis-Hastings ratio;
c: Ymiss(t) from p(Y(t)|X(t−1),Θ(t));
d: Xmiss(t) from p(X(t)|ψ(t)).
6: end for
Steps 5a to 5d of Algorithm 4.1 are repeated for allΘandψ terms, allYmissmiss- ing responses and allXmissmissing covariates, until convergence is obtained for all these terms altogether. The methods for convergence check (i.e. trace plots containing mean estimates and standard errors) are demonstrated in Section 4.7.
Calculating the conditional terms in Steps 5a to 5d of Algorithm 4.1 can be a technically demanding and complex task. WinBUGSandOpenBUGSprogrammes (Lunn et al. (2000); Spiegelhalter (2003); Spiegelhalter (2009)) make this com- puting task much simpler for the user. InWinBUGS, only the priors, p(Θ) and
p(ψ), and the likelihood, p(X |ψ) and p(Y |X,Θ), are needed to be specified,
and the conditional terms are sampled from the specified priors and likelihoods automatically.
In theOpenBUGSprogram (Spiegelhalter, 2009), the Gibbs sampling (Geman and Geman, 1984) is the main sampler of the fully Bayesian imputation, in which the component imputation involves various methods, including rejection sampling
CHAPTER 4. MISSING DATA THEORY, METHODOLOGY AND APPLICATION104 and Slice sampler. Further details about Slice sampler and rejection sampling can be found in Ntzoufras (2009). In this research, OpenBUGS program was used because the environment was easy to use for writing Bayesian imputation programs (Murphy, 2007).
4.5.3.2 Multiple Imputation by Chained Equations - First Stage
Multiple imputation by chained equations creates many copies of the fully ob- served data with varying imputed values for missing data. It is a two-stage imputation model, as the missing data are imputed by an imputation model, and statistical inferences are implemented on imputed data sets with a substan- tive model. Section 4.5.3.3 discusses how the models on the imputed data sets are combined. In this section, we show how the multiple imputation by chained equations can be used to carry out multiple imputation.
The multiple imputation by chained equations (MICE)(Buuren, 2012) is an im- plementation of fully conditional specification (FCS), which imputes missing data on a "variable-by-variable" basis, for T iterations, P number of variables in the data set and W imputed data sets. The MICE imputation deals with parameters for substantive model at iterationt of the Gibbs sampler, denoted asΘ(wt,)p, and parameters for imputed model denoted asψw(t,)pfort =1, . . . ,T,w=
1, . . . ,W. This imputation deals with each parameter sequentially, which are
Θw,p,ψw,p,p=1, . . . ,P,w=1, . . . ,W. The missing data Xw,p,miss is sampled from Xw(t,)p,miss ∼P(Xw(t,)p,miss|ψw,p) for t =1, . . . ,T,w =1, . . . ,W,p=1, . . . ,P. With im- plementation of FCS and Gibbs Sampling (Geman and Geman (1984);Buuren (2012)), the MICE sampling procedure can be described with the following equations and the steps of MICE algorithm for imputation of multivariate miss- ing data are listed as Algorithm 4.2 of the first stage of modelling (Buuren, 2012).
Algorithm 4.2First Stage of MICE Modelling Algorithm
1: forImputationw=1, . . . ,W do
2: Specify the imputation model Xw,miss, p(Xw,miss|ψw) and the likelihood function, p(Yw|Xw,Θ)
3: InitialiseYw,missasY
(0)
w,miss.
4: forAll parameters p=1, . . . ,Pdo
5: InitialiseXw,p,miss asX (0) w,p,miss. 6: InitialiseΘw,pasΘ (0) w,p. 7: Initialiseψw,pasψw(0,)p. 8: end for 9: end for 10: forImputationw=1, . . . ,W do 11: fort=1, . . . ,T do 12: Yw(t,miss) ∼P(Yw(t,miss) |. . .); 13: forp=1, . . . ,Pdo 14: Xw(t,)p,miss∼P(Xw(t,−p,1miss) |ψw,p); 15: Θ(wt,)p∼P(Θ(wt,)p|. . .); 16: ψw(t,)p∼P(ψ (t) w,p|. . .); 17: end for 18: end for 19: end for 20: forw=1, . . . ,W do
21: Upon convergence, obtain the last imputed data set: Yw(,Tmiss) as the wth imputed data set,Yw, and proceed to Second stage MICE algorithm.
22: end for
gram (R Version 3.3.0). This program calculates posterior probabilities based on generalized linear models. For binary data response, we specify a logistic regression model; for nominal variable with more than two levels, we specify a polytomous regression model, and for continuous data, we specify a linear regression with prediction method. Details of this program can be referred to theR program manual (R Development Core Team, 2008) about micepackage (Buuren and Groothuis-Oudshoorn, 2011).
4.5.3.3 Multiple Imputation by Chained Equations - Second Stage
The second stage of modelling fits the substantive models on imputed data sets, followed by combining estimates of the substantive models through Rubin’s rule
CHAPTER 4. MISSING DATA THEORY, METHODOLOGY AND APPLICATION106 (Rubin, 1987). Through the substantive model for each wth of the W imputed data sets,w=1, . . . ,W, a set of parameter estimates,Θˆw and its covariance ma- trix,Vˆw=Var(Θˆw), are obtained for variable Θp. Estimates and variances from W imputed data sets are combined by Rubin’s rule (Rubin, 1987). Rubin’s rule is applicable to imputed data sets under MAR, ignorability and normality as- sumptions (Allison, 2003).
Suppose W sets of estimates are obtained from analysis of W imputed data sets for the variableΘp, denoted as the estimate vector Θˆw,w=1,2, . . . ,W, the combined mean estimate vector forW imputed data sets, Θ¯W, is calculated by Equation 4.9: ¯ ΘW = 1 W W
∑
w=1 ˆ Θw. (4.9)Regarding the calculation of total combined covariance matrix, firstly we explain the calculation of the combined within-imputation covariance matrix,V¯W, and then the combined between-imputation covariance matrix,B¯W. Finally, we ex- plain the calculation of the total combined covariance matrix,T¯W. At this stage, we utiliseW number of covariance matrices,Vˆ1, . . . ,VˆW, that are associated with estimate vectorsΘˆ1, . . . ,ΘˆW respectively.
The combined within-imputation covariance matrix,V¯W, is calculated as in Equa- tion 4.10, which is basically the mean of variances forΘˆw, w=1, . . . ,W, fromW
imputed data sets:
¯ VW = 1 W W
∑
w=1 ˆ Vw. (4.10)The combined between-imputation covariance matrix, B¯W, is calculated as in Equation 4.11: ¯ BW = 1 W−1 W
∑
w=1 (Θˆw−Θ¯W)(Θˆw−Θ¯W)T. (4.11)Hence, the total combined covariance matrix, T¯W, is calculated as in Equation 4.12: ¯ TW =V¯W+ 1+ 1 W ¯ BW, . (4.12) (Rubin, 1987) and Rubin’s combination rule hence provides an unbiased estimate of the total combined covariance matrix.
Wald’s test is used to determine whether a variable is significantly related to responses. For combined estimates, Wald’s test statistics are adopted for testing a certain variable,Θp, which containsk0components to be tested. SupposeΘ¯W is the mean estimate vector for Θp over W imputed data sets in Rubin’s rule equations, Θ¯0 is the vector of null values for testing Θp, V¯W and B¯W are com- bined within-imputation covariance matrix and combined between-imputation covariance matrix respectively in Rubin’s rule equations, the Wald’s test statistic,
ω(Θ¯W), is calculated as follows:
ω(Θ¯W) =
(Θ¯W−Θ¯0)TV¯W−1(Θ¯W−Θ¯0)
(1+r)k0 , (4.13) wherek0is the number of components being tested,r= (1+1/W)trace(B¯WV¯W−1)/k0. The p-value by F distribution is then evaluated, and the corresponding p-value is stated as follows:
P[Fk0,l>ω(Θ¯W)], (4.14) whereFk0,lis a random variable of F distribution withk0andldegrees of freedom.
Fork0(W−1)>4,lis defined as follows: l=4+ [k0(W−1)−4]1+z r 2 ,z= 1− 2 k0(W−1) . (4.15) Alternatively,l= (W−1)(k0+1)(1+1/r)2/2ifk0(W−1)≤4.
CHAPTER 4. MISSING DATA THEORY, METHODOLOGY AND APPLICATION108 (Li et al., 1991) The Wald’s test will also be applied in the backward elimination process with Rubin’s rule in logistic regression model, log-linear analysis model, item re- sponse theory model and latent class analysis model.
The algorithm of the second stage of the MICE imputation is described as Algo- rithm 4.3.
Algorithm 4.3Second Stage of MICE Variable Selection Algorithm (Combining Estimates)
1: whileAn insignificant covariate exists in a substantial backward elimination modeldo
2: forw=1, . . . ,W do
3: Fit the substantial model with everyYw, and generateΘˆw,VˆWas results.
4: end for
5: CalculateΘ¯W =W1 ∑Ww=1Θˆw.
6: CalculateV¯W =W1 ∑Ww=1Vˆw.
7: Obtain the between-imputation variance B¯W and total variance T¯W by Rubin’s Rule.
8: Perform Wald’s test withΘˆw,Θ¯W,V¯W, B¯W and T¯W on each covariate and determine which insignificant one (at 5% significance level) to discard by the highest p-value.
9: Discard the insignificant covariate with the highest p-value. 10: end while
To determine which covariates to be included in the imputation model, poly- choric correlation plots among complete cases are adopted before imputation of missing values. Generally, covariates that yield correlation value with any other covariates of 0.3 or more are included in the imputation model. Referring to Figures 3.3 to 3.5, most covariates yielded high correlation values with at least one other covariates. In addition, percentage tables and box plots in Chapter 3 reflected that most variables were related to drug response variables. As such, all covariates were included in the imputation model of MICE imputation.