In this section we conduct a simulation study to compare, in terms of size and power, the four score test statistics HS, HSC, HN B and HQL. For studying the properties of the
statistics in terms of empirical size we generate count data from Poisson, negative bino- mial and lognormal-Poisson mixture distributions under the hypothesis of homogeneity. We assume random effect is the intercept (Zij = 1).
Two sets of data are simulated for each distribution of the response variable as- suming homogeneous and heterogeneous inner group sizes (= ni) with different num-
ber of groups/individuals (k) according to the variance (D) of the distribution of the group-specific random effect of the response variable and for different values of the over-dispersion parameter c. The samples are comprised of k = 10, 20, 50, 100 individu- als/groups with ni = 5 observations in the homogeneous group and ni distributed uni-
formly between 5 and 20 in the heterogeneous group. The values of the over-dispersion parameter c considered are 0.10, 0.22, 0.40, 0.67, 0.91 and 1.25. We generate 10,000 sam- ples from each experiment in computing the nominal levels and power. The following log-linear model for the response variable is assumed (see Jacqmin-Gadda and Com- menges (1995))
log(µij) = 0.8x1ij+ 0.5x2i− 0.5, (3.18)
for i = 1, 2, . . . , k and j = 1, 2, . . . , ni. The variable x1 is subject-specific and x2 is
group-specific and are simulated according to a standard normal distribution.
For drawing samples and for estimating the maximum likelihood estimates of the regression and dispersion parameters of interest under the null hypothesis, different R functions are applied. To simulate correlated data, we add a group-specific random intercept in the model for the response variable which is αi = α + D1/2vi, where vi is
standard normal. Therefore, the random effects are normally distributed with mean α and variance D.
3.1 displays the computed nominal levels for the four tests when data are generated from the Poisson distribution according to the variance of the distribution of the group- specific random effect under the hypothesis of homogeneity, that is, D = 0. None of the statistics maintain level, although the level property, in general, improves as the sample size increases, except for the statistics HQL. However, HN B performs the
best in maintaining level. The performance of the statistic HQL is the worst, severely
underestimating the nominal levels. There does not seem to be any difference in level properties of all the statistics between cases when the groups have homogeneous sample size and the case when the sample sizes are not equal.
Tables 3.2 and 3.3 display the estimated Type I error for the four tests when data are simulated from the negative binomial distribution under the hypothesis of homo- geneity with common over-dispersion parameter c. From the results in Tables 3.2 and 3.3 it is evident that both the statistics based on the generalized linear mixed model (GLMM) framework perform the worst, severely overestimating the level as the number of groups increases. This property is even worst when the number of groups as well as the over-dispersion are large. The other two statistics HN B and HQL, in general,
show conservative behavior, although as the number of groups increases the estimated levels become closer to the nominal levels. Level properties of both of these statistics are similar.
We then generated count data with over-dispersion from the Log-normal-Poisson mixture model under the hypothesis of homogeneity with common over-dispersion pa- rameter c. The mean and variances of the log-normal distribution assumed were m = log(µij) − 12log(1 + c) and σ2 = log(1 + c) as used by Paul and Banerjee (1998), where
log(µij) is given by (3.18) and c is the common over-dispersion parameter. Tables 3.4
and 3.5 display the estimated Type I error of the four tests. From Tables 3.4 and 3.5 we see that the overall properties of all of the four statistics remain almost the same as those shown in Tables 3.2 and 3.3 when data were simulated from the negative binomial distribution, except when k and c are large (k ≥ 50 and c ≥ .67) in which case both the
statistics HN B and HQL show liberal behavior.
Our second objective is to compare power of the four tests when data are gen- erated from heterogeneous count data models. Here we first consider data from the heterogeneous negative binomial model. As for size here also we consider nominal levels 0.10, 0.05 and 0.01. We computed power for k = 20 and 50 with common c = .10, .40 for D = 0, .05, .10, .15, .20. Table 3.6 presents computed power of the four tests.
From the results in Table 3.6 we first discuss results for k = 20 and c = .10. As the value of D increases, the power increases for all the statistics and they all show similar power. Note that at these values of k and c all the four statistics have similar level property (and they all reasonably hold level) and they also have similar power. This indicates that if all the four statistics have similar level, they will have similar power.
Now we consider k = 20 and c = .40. As the statistics HS and HSC produce highly
inflated Type I error, their powers are also overestimated. Power properties of the other two statistics HN B and HQLare similar. However, the power of HN B and HQLincreases
faster as D increases. For example, at α = .01, the estimated level of HS and HSC is .06
and for HN B and HQLis .007. The corresponding empirical power for the four statistics
HS, HSC, HN B and HQL for D = .2 are .70, .69, .60 and .60, respectively. Note that
empirical level 0.007 for HN B and HQL is close to the nominal level of α = .01, whereas
the empirical levels for both HS and HSC are about six times the nominal level.
Next we consider k = 50. Generally, as c increases power increases and also as the number of observations in each group increases power increases. The general properties of HS and HSC for k = 50 and c = .10 and .40 are similar to those for k = 20 and
c = .40. As D increases (D > .1) power of all of the statistics become almost identical,
although empirical levels of the statistics HN B and HQL are close to the nominal and
those of HS and HSC are highly inflated.
The power study was extended for the situation in which data are generated from heterogeneous log-normal-Poisson mixture distribution. The results are given in Table 3.7. The overall finding from the results in Table 3.7 seem to be similar to those in Table
3.6 when data are generated from the heterogeneous negative binomial model.
In summary, as k, c and ni increase, the power increases for all the statistics. The
statistics HS and HSC, in general, show highly inflated level properties. The statistics
HN B and HQL show some conservative level properties, however, as the values of c and
k increase, empirical levels become closer to the nominal. The power of the statistics HS
and HSC are, in general, larger than those of HN B and HQL which is expected. What
is interesting is that as D increases (D > .1), the power of all of the statistics become almost identical, although empirical levels of the statistics HN B and HQLare close to the
nominal and those of HS and HSC are highly inflated. The power of both the statistics
HN B and HQL are very similar in all the cases studied.
We extended this simulation study of the properties of the four statistics in terms of empirical size and power to situations where the over-dispersion parameter c is not the same for all groups. For this we generated data from the heterogeneous negative bino- mial and Log-normal-Poisson mixture distributions with heterogeneous over-dispersion parameter c(.10 ≤ c ≤ 1.0). The results for size and power are given in Tables 3.8 and 3.9. The overall conclusion of the level and power properties of the four statistics remain the same as those for homogenous c.
The level and power properties of all the statistics, in general, remain similar irre- spective of which mechanism of over-dispersion is used to generate count data. This also seems to be true irrespective of whether the over-dispersion parameter c is varying or constant.