Simulation Extrapolation Method

Here we describe the SIMEX method to correct the bias due to measurement error. More details are available in Carroll et al. (2006). Although the theory of the SIMEX method is not trivial, an example from simple linear regression can well illustrate the idea of this method. Suppose the response variableY and the covariateX is modeled as

where has mean 0. Here X is normal distributed with variance σ_x2. If replacing X with its observed measurementW, modeled by W =X+U, where U follows normal distribution with mean 0 and variance σ_u2 , and is independent of and X, then the resulting least squares estimator ˆβ_x∗ for βx converges in probability to

β_x∗ = σ

2 x σ_x2 +σ_u2βx,

instead ofβx. To see how the bias may be related to the degree of measurement error

inX, we perturbW by adding additional error to createW(b, λ) =W+√ λU_b where U_b is independently generated from a N(0, σ_u2) distribution. Intuitively, if regressing Y over the perturbed version W(b, λ), then the resulting estimator ˆβx(b, λ) would

converge in probability to

β_x∗ (b, λ) = σ

2 x

σ2_x+ (1 +λ)σ_u2βx.

This expression indicates the dependence of the asymptotic bias on the magnitude of measurement error - the less degree of measurement error (equivalently, a smaller λ), the smaller asymptotic bias. In particular, if λ shrinks to 0, ˆβx(b,0) recovers the

naive estimator ˆβ_x∗ ; if λtakes value -1, then the limitβ_x∗ (b,−1) is identical to the true parameter βx.

We consider two practical cases for the parameters in Σu_i: (i) the parameters

in Σu_i are given as fixed values; and (ii) the parameters in Σu_i are not known, but

replicate measurements of W_i are available.

Suppose that one could estimate regression parameterθ by solving an estimat- ing equation given by

0=

n

X

i=1

ϕ(θ;Yi,Xi). (2.3)

the estimates and associated variances of the parameters θ can be obtained by the following SIMEX algorithm.

2.2.3.1 Case I: The parameters in Σu_i are given as fixed values.

1. Simulation Step

We generate the pseudo errors U_bi ∼ MVN(0,Σu_i) for i = 1, . . . , n, b =

1, . . . , B. For each λ∈ Λ, let the pseudo data be given by

Wi(b, λ) =Wi+λ

1 2_Ubi.

Note that E(W_i(b, λ)|X_i) = X_i and Var(W_i(b, λ)) = Σw_i(b, λ) = Σx+ (1 + λ)Σu_i. When λ=−1,Σw =Σx. Hence, the mean square error, E[(Wi(b, λ)− Xi)2|Xi], converges to zero as λ → − 1. This implies that Wi(b, λ) is the best

measurement of X_i. This is the most important property of the pseudo data (Carroll et al., 2006).

2. Estimation Step

Given λ and b, we obtain the estimate bθ(b, λ) by solving equation (2.3) with

X_i replaced by W_i(b, λ). The corresponding variance estimate, Ωb(b, λ), is the diagonal elements of the observed information matrix given by

" − n X i=1 ∂ ∂θ0 ϕ(θ;Yi,Wi(b, λ))|θ=bθ(b,λ) #−1 .

By averaging over b, we obtain

b θ(λ) =B−1 B X b=1 b θ(b, λ)

and b Ω(λ) = B−1 B X b=1 b Ω(b, λ). Define bS(λ) = (B − 1)−1 B P b=1 b θ(b, λ)− bθ(λ) 2

, the variance for the estimator b

θ(b, λ) is given by

b

Ω(λ)− Sb(λ).

3. Extrapolation Step

Fit these average estimates to an extrapolation function of lambda and extrap- olate bθ(λ) back to the case of no measurement error (i.e., λ=−1) to yield the SIMEX estimateθb_simex. The variance estimate for bθ_simex can be obtained by extrapolating Ωb(λ)− bS(λ) back to λ=−1.

The most commonly used extrapolation functions are linear extrapolation func- tion, quadratic extrapolation function and rational linear extrapolation func- tion. The quadratic extrapolant function in the SIMEX method is generally a safe choice for most regression models (He et al., 2007).

2.2.3.2 Case II: The parameters in Σu are not known, but replicate mea-

surements of W_i are available.

Consider the case where the measurement error covariance matrices are not known but replicate measurements ofWi are available. The procedures of the SIMEX algorithm

described above can be applied except for the data simulation step. Devanarayan and Stefanski (2002) introduced this variation and named it empirical SIMEX.

Suppose we have k_i ≥ 2 replicate measurements denoted by {W_i1, . . . ,W_ik

i}

for every subject i such that

whereU_ij are mutually independent from each other and independent of{Y_i,X_i}. For fixedi, U_ij are independent and identically distributed random errors such that U_ij iid

∼ MVN(0,Σu_i). Without knowing the covariance matrix Σu_i, we cannot generate

pseudo errors directly. However, we can obtain them by taking linear combination of the replicated measurements.

The empirical SIMEX algorithm is as follows

1. We generate z_b,i,jiid∼ N(0,1), i= 1, . . . , n, j = 1, . . . , k_i, b= 1, . . . , B. We define ¯

z_b,i,. =Pk_i

j=1zb,i,j/ki and

c_b,i,j = _q zb,i,j− z¯b,i,.

Pk_i j=1 zb,i,j − z¯b,i,. 2 , where Pk_i j=1cb,i,j = 0 andP k_i j=1c2b,i,j = 1.

2. For i= 1, . . . , n,j = 1, . . . , ki, b= 1, . . . , B and each λ∈ Λ, we define

W_i(b, λ) = ¯W_i.+ λ ki 1₂ ki X j=1 c_b,i,jW_ij, where ¯W_i. = Pk_i

j=1Wij/ki. It is easy to see that E(Wi(b, λ)|Xi) = Xi and

Var(W_i(b, λ)) =Σw_i(b, λ) = Σx+ (1 +λ)Σu_i/ki.

This pseudo data have the same important property as the SIMEX algorithm defined in the previous section. That is, as λ → − 1, E[(W_i(b, λ)− X_i)2|X_i] converges to zero. We estimate bθ(b, λ) by solving equation (2.3) with X_i re- placed by Wi(b, λ) for each b and λ. Then, we averaged over b to obtain

b

θ(λ) =PB

b=1θb(b, λ)/B.

For practical use, choosing B = 50, 100 or 200, and taking Λ to be equally cut points of interval [0,1] or [0,2] with M = 5, 10 or 20 can often lead to fairly reasonable SIMEX estimates (Carroll et al., 2006).

2.2.3.3 Asymptotic Normality of SIMEX Estimate

The adaptive LASSO regularized IPW method is utilized to select the variable and estimate the covariate coefficients under the AFT setting (2.1). Here we assume that all the covariates are subject to measurement error and the SIMEX method is applied to adjust the effect of measurement error on variable selection on AFT models. For each λ and b. We obtain estimates, βb_x(b, λ), by minimizing

`(β_x(b, λ)) = 1 2 n X i=1 Y_π(i)− W_i(b, λ)_π(i)0 β_x(b, λ)2 +γ p X j=1 v_j βxj(b, λ) .

For any givenλ, by applying Theorem 2 of Zou (2006), we have

√

nβb_x(b, λ)− β_x(b, λ)

→ _d MVN(0,C(b, λ)),

where β_x(b, λ) is the true unknown adaptive LASSO regularized IPW parameter. C(b, λ) is the variance parameter. We calculate the average of these B estimators, b

β_x(λ) =B−1PB

b=1βb_x(b, λ), and let C(λ) = B−1 PB

b=1C(b, λ)). According to Slut-

sky’s theorem, we have

√

nβb_x(λ)− β_x(λ)

→ _dMVN(0,C(λ)).

Let βb_x(Λ) = vec{βb_x(λ);λ ∈ Λ}, β_x(Λ) = vec{β_x(λ);λ ∈ Λ} and Γ(Λ) = diag{C(λ), λ∈ Λ}, we have √ n b β_x(Λ)− β_x(Λ) → _dMVN(0,Γ(Λ)).

Assume that the exact extrapolation functionG(ζ;λ) is known in the extrapo- lation step to fitβb_x(λ), where ζ is d-dimensional vector of parameters. Let bζ be the least squares estimator. Let G_ζ(ζ;λ) = (∂/∂ζ)G0(ζ;λ),

s(ζ) = (G_ζ(ζ;λ₁)G_ζ(ζ;λ₂)· · ·G_ζ(ζ;λ_M))

and A(ζ) =s(ζ)s0(ζ) be the d× d matrix. Then by the similar argument to that of Carroll et al. (1996), we obtain

√

n(bζ − ζ)→ _d MVN(0,Q(ζ)),

where Q(ζ) = [A(ζ)]−1s(ζ)Γ(Λ)s0(ζ)[A(ζ)]−1 is a d× d matrix. Letting λ = −1 leads to the SIMEX estimator βb_x =G(bζ;−1). Therefore, obtain

√

n(βb_x− β_x)→ _dMVN(0,G_ζ0 (ζ;−1)Q ζ)G_ζ(ζ;−1)

.