I ntroduction

The estimating equations (1.3) - (1.4) and (1.6) - (1.7) have been derived under the assumption that the data are normally distributed, but they can still be used to estimate the parameters when the data are not normally distributed. In practice we can never be sure the data are normal, so we are usually in the position of using a method derived under normality when normality does not necessarily hold. Generally in this circumstance, the estimand (which is the quantity that makes the expected value of the estimating equations zero) is no longer T0, the true value of the parameters, but the true value plus an unknown bias that for non-robust methods like (1.3) - (1.4) and (1.6) - (1.7) may be infinite. Even if the bias is found to be finite in a particular application, non-robust methods can be extremely inefficient when the model distribution (in this case normality) does not hold; see Stahel and Welsh (1992). We would prefer to use a robust method for which the potential bias is always bounded, that is reasonably efficient when normality holds, and also more efficient than methods derived under normality when normality does not hold.

Important early research into robust methods for general models has been gathered into the books by Huber (1981) and Hampel et al. (1986). Various approaches are discussed by these authors: the most useful seems to be M-estimation,

n

where the log-likelihood, ^ log(f(yi, x)), is replaced by a more general function i=l

n

denoted p(yi, x). M-estimation may also be treated as replacing the estimating i=l

n

equation from the log-likelihood, ^ 3log(f(yi, x))/3x = 0, by a more general function i=l

n

denoted ^ \j/(

yu

the case. We will be concentrating on bounded \|/ functions, because the influence of outlying observations on the size of the estimate is thereby bounded.

Several authors have applied the principles of M-estimation to the specific problem of calculating robust estimates of variance components. Some of the different approaches that have resulted are listed below. Stahel and Welsh (1992) discuss many of these methods, and also suggest some modifications, for the balanced two- component fully random model defined in Section 1.2.

Fellner (1986) defines a robust REML by robustifying Harville's (1977) REML algorithm: this method will be discussed in Section 3.4.

Rocke (1983) robustifies ANOVA estimates for the two-component fully random model described in Section 1.2. He calculates M-estimates of a + ßj and a and uses them to create pseudo-observations, where large observations are replaced with values closer to the bulk of the observations. He then uses the pseudo-observations to calculate ordinary ANOVA estimates of the variance components. One drawback of this method is that it robustifies a method that has a non-zero probability of producing estimates outside the parameter space. Furthermore, it is unclear how this method should be extended to unbalanced models.

Hampel et al. (1986) introduce optimal B-robust estimators, which cannot be improved upon simultaneously in terms of asymptotic bias and asymptotic variance. First they deal with one-dimensional parameters (sec. 2.4), then multidimensional parameters (sec. 4.3) and as a special case of the latter, the parameters of a linear model (sec. 6.3). Stahel and Welsh (1992) show that in the balanced two-component fully random model, the parameters a, 0i + m_102 and 02 are orthogonal. The optimal B-robust estimator is then found by applying a \\f function to the entire left—

g

hand side of the estimating equations ^ 3log(f(yj, x))/3t = 0, where T represents the j=l

vector of orthogonal parameters. However, considerable complications arise in extending this method to unbalanced data because of the non-orthogonality of variance components in such models.

Stahel and Welsh (1992) also discuss the possibility of replacing the normal density in (1.2) by an arbitrary density. In practice the most popular density chosen for this purpose has been Student's t on v degrees of freedom: see Lange et al. (1989). The degrees of freedom parameter means that there is one more parameter to estimate compared to normal maximum likelihood, but Lange et al. suggest that rather than maximising the t log—likelihood with respect to a, 0 and v, that an optimal value of v be chosen via a grid search, or even by fixing v a priori at about 3 or 4 if the sample size is rather small. James et al. (1993) defined and named an analogous t- REML, for the linear regression model with heteroscedastic errors.

But, as Lange et al. point out, the t distribution does not cope with all departures from normality, in particular with extreme outliers and asymmetric error distributions. Huggins and Staudte (1994) offer more arguments against using t modelling. In particular, the t distribution models the data vector as it stands, outliers and all, rather than concentrating on the central part of the data, which is probably of most interest. Thus, if the data contains a single outlier due to a typing error, the whole t distribution changes to take account of this error.

Furthermore, it is important to make the distinction between the "independent t" approach of Lange et al., where yj ~ tmj(Xj(X, Vj, v), and the "multivariate t" approach where y ~ tn(Xa, V, v) but V may be block diagonal. Breusch and Robertson (1993, p.24) show that "a statistical model based on a single realization of

the multivariate t distribution is only capable of modelling those features of the data that would have been accounted for by a multivariate normal distribution. In particular, the multivariate t model can not explain either observed leptokurtosis or dynamic heteroskedasticity in a data set." They conclude that the multivariate t model is of "limited interest".

Huggins (1993a, b) defines robust maximum likelihood estimators by replacing the quadratic function (—l/2)(yj - XjOt)TV j1 (yj - Xjtx) in (1.2) by a more general

- 1/2

function —Pj(Vj (Yj “ Xj(x)), and maximising this new log-likelihood in the usual

way. In Section 3.2 we place Huggins' method in a wider context by calling it Robust ML I, and defining a companion estimator, Robust ML II. We then extend these two estimators to obtain two robust restricted maximum likelihood estimators, denoted Robust REML I and II.

In Section 3.3 we return briefly to non-robust estimation, showing how to apply the EM algorithm to REML estimation of variance components, and showing that Harville's (1977) algorithm for solving (1.6) is equivalent to the EM algorithm. Then Section 3.4 is devoted to Fellner's (1986) method for calculating robust REML estimates, which he developed by robustifying Harville's algorithm. We show that Fellner's method can be derived directly from (1.6), thereby creating a full set of links between EM, REML, Harville and Fellner. Finally, in Section 3.5 we apply some of the estimation methods discussed to the wheat data and the metallic oxide data.