Covariance Structures

The error covariance structure of the model in equation 2.21 allowed for het- eroscedasticity but not for autocorrelation between the composite residuals. This section reviews the most common types of covariance matrices that can be imposed on the error components.

Suppose that the number of time points Tij is equal to 3 for every individual

i and that a two-level longitudinal model is considered. The following structures can be imposed (Singer and Willett, 2003; Diggle et al., 2002; Fitzmaurice et al., 2004):

- Unconstrained: It imposes no specific structure on the error covariance matrix Σr. This covariance matrix is of the form:

Σunc_r =    σ₁2 σ21 σ22 σ31 σ32 σ32   .

Each diagonal term, the variances, and off diagonal term, the covariances, of Σunc

r has their own value and they are parameters of the model to be

estimated. Models estimated imposing the unconstrained, or unstructured, error covariance will have smallest deviance due to the larger number of esti- mated parameters. With T occasions there will be T ×(T +1)₂ extra parameters.

Fitzmaurice et al. (2004) stated this structure is not advisable for highly un- balanced data sets or data sets with relatively few individuals. Singer and Willett (2003) advised comparing models fitted with imposed unstructured correlation using the AIC and BIC criteria.

- Compound Symmetry: Also known as the exchangeable structure or uni- form. This is the usual structure assumed in the longitudinal multilevel model when only the random intercept is considered. In other words, it is assumed that the variance at any time point is the same, as well as that the covariance between any pair of time points is the same. For the data set being considered in this section, this covariance matrix is of the form:

Σexch_r =    σ2 u+ σe2 σ2 u σ2u+ σe2 σ2 u σu2 σu2+ σ2e   .

With this structure the correlation between any pair of residuals will also be the same and equal to ρ as in equation 2.5. In addition, if a random slope is fitted but ˆσu01 and ˆσu12 are small the exchangeable structure may hold.

- Heterogeneous Compound Symmetry: This is an extension of com- pound symmetry but not assuming homoscedasticity along the diagonal terms of Σr. In addition the assumption of equal covariance between pairs

of residuals is also relaxed.

Σhexch_r =    σ2 1 σ2σ1ρ σ22 σ3σ1ρ σ3σ2ρ σ32   .

This structure has a constant autocorrelation parameter ρ also estimated by the model.

- Autoregressive: This is the first-order autoregressive correlation structure, also called exponential for continuous time data (Schabenberger and Pierce, 2001). The variances are assumed constant across time and equally spaced pairs of responses have the same covariance (Fitzmaurice et al., 2004) which depends on the lag between them. It causes the “band-diagonals” of Σr to

be the same. The main diagonal expresses a constant variance term and the other diagonals are determined by:

The Σr matrix under this structure has the form: Σar_r =    σ2 σ2_{ρ σ}2 σ2_ρ2 _σ2_{ρ σ}2   .

This structure assumes that the correlation between pairs of residuals dimin- ishes for larger lags. The model estimates only two variance components. However, the degree to which the correlation diminishes is determined by a constant ρ.

- Heterogeneous Autoregressive: This is an extension of autoregressive structure but not assuming homoscedasticity along the diagonal terms of Σr, just as with the heterogeneous exchangeable structure. In addition the

terms of the off diagonals are determined by: Cov(rt, rt0) = σ_r

tσrt0ρ

lag _{lag = 1, 2... .}

Under this structure the covariance matrix of the residuals has the form:

Σhar_r =    σ₁2 σ2σ1ρ σ22 σ3σ1ρ2 σ3σ2ρ σ32   .

This is more flexible than Σar_r .

- Toeplitz: This structure has similar characteristics to the Σar

r . However, the

elements of the band-diagonals are not forced to reduce by a fixed fraction (Singer and Willett, 2003). This structure still considers the main diagonal to be constant and the covariance matrix under this structure has the form:

Σtoep_r =    σ2 σ1 σ2 σ2 σ1 σ2   .

This imposes that pairs equally separated in time have the same correlation (Fitzmaurice et al., 2004) and is only appropriate for equally spaced data. The different variance components are parameters of the model to be esti- mated. Compared to those with Σar

r , models with the Toeplitz2 structure

will have less residual degrees of freedom.

Two more general covariance structures that deserve mentioning are the following.

- Spatial Power: Also known as Markov Structure (Khattree and Naik, 1999). This is a reparameterisation of the exponential correlation structure, which, as mentioned earlier, is equivalent to a continuous time autoregressive structure. The exponential correlation structure can be written as:

Cov(rt, rt0) = σ2exp

 |t − t0_|

−φ 

∀t 6= t0 .

This structure, like the AR(1), imposes that the correlation between any pairs of residuals will be smaller if measured further apart (Diggle et al., 2002). Furthermore, the larger the value of 1/φ the faster the correlation decays towards zero as the distance between the pairs of residuals increases. The reparameterisation for the Spatial Power structure involves setting

ρ = exp −1 φ



and expressing the covariance terms as:

Cov(rt, rt0) = σ2ρ|t−t

0_|

.

This is a direct generalization of AR(1) for unequally spaced data that takes into account the distance between the T occasions by powering ρ by |t − t0|. The name for this structure, spatial power, is justified as it is usually applied to studies of spatial processes (Khattree and Naik, 1999). For the data set considered in this section, the Spatial Power covariance matrix is also of the form: Σpow_r =    σ2 σ2ρ1 σ2 σ2ρ2 σ2ρ1 σ2   .

- General Linear: Assuming that the residual covariance matrix can be expressed as a linear function of θ, the general linear covariance structure (SAS Institute Inc,Version 8, 1999; Khattree and Naik, 1999; Jennrich and

Schluchter, 1986) and (Pourahmadi, 2007) is of the form: Σgen_r = θ0A0+ θ1A1+ ... + θkAk,

where the matrices Ak are known symmetric matrices and the parameters

θk are unknown and unrelated covariance parameters to be estimated by the

model (Khattree and Naik, 1999). The known matrices Ak can be set to

represent any of the known structures or any desirable structure with the requirement that Σgen

r must be a positive definite matrix. For example, the

compound symmetry structure could be expressed as:

Σexch_r =    σ_u2+ σ_e2 σ_u2 σ2_u+ σ_e2 σ_u2 σ_u2 σ_u2+ σ2_e    = σ_u2    1 1 1 1 1 1 1 1 1   + σ 2 e    1 0 0 0 1 0 0 0 1   .