Model Evaluation and Testing
5.5 Inference with Finite Sample Corrections
If f (yi| µi) is the Poisson density this yields
∂L ∂α α=0= n i=1 1 2g(µi)µ −2 i {(yi− µi)2− yi}, (5.54)
which is exactly the same as the second term in (5.38). The first term,∂L/∂β|α=0 is also the same as in (5.38), and theLMtest statistic isTLM, given in (5.42).
The approach of Cox (1983) demonstrates that TLM in (5.42) is valid for
testing Poisson against all local alternatives satisfying (5.35) and (5.36), not just the Katz system. The general form (5.52) for the density under local alternatives is clearly related to the information matrix equality. In section 5.6.5 we make the connection between the Cox test and the information matrix test.
5.5 Inference with Finite Sample Corrections
A brief discussion of small-sample performance of hypothesis tests in the Pois- son and negative binomial models is given by Lawless (1987b). He concluded
that the LRtest was preferable for tests on regression coefficients, although none of the methods worked badly in small samples. There was, however, con- siderable small-sample bias in testing dispersion parameters.
The standard general procedure to handle small-sample bias in statistical inference is the bootstrap, proposed by Efron (1979) and presented in the next subsection. This method has to date not been widely applied to count models. One potential complication is that the bootstrap requires resampling from an
iiddistribution, but the errors in cross-section count data models are typically not identically distributed.
The main application of small-sample corrections for count data analysis has been a method quite different from the bootstrap, one proposed by Dean and Lawless (1989a) forLMtests of overdispersion. This method has been applied in several studies, which find in simulations that the improved size performance is small except if sample sizes are small, say less than 30 observations.
5.5.1 Bootstrap
The bootstrap, introduced by Efron (1979), is a method to obtain the distribution of a statistic by resampling from the original data set. An introductory treatment is given by Efron and Tibsharani (1993). An excellent treatment with emphasis on common regression applications is given by Horowitz (1997). Here we focus on application to cross-section count data regression models, particularly the PoissonPMLE, using the bootstrap pairs procedure under the assumption that (yi, xi) isiid.
Reasons for performing a bootstrap in estimation and statistical inference include weaker stochastic assumptions, simpler computation, and potentially better small-sample performance. The first two reasons are often compelling reasons for using the bootstrap in applied work even if small-sample perfor- mance gains are not achieved. An example of using the bootstrap under weaker stochastic assumptions has been given in section 3.2.6, where the standard error of the PoissonPMLEwas obtained under the assumption that (yi, xi) isiidand
E[yi| xi]= exp(xiβ). HereV[yi| xi] is not specified, and the distributional as-
sumptions are similar to those made in obtaining the robust sandwich standard errors. Examples of simpler computation include obtaining the distribution of
r( ˆθ), in applications in which r(·) is a complicated function of θ, by bootstrap
rather than the delta method given in section 2.6.2; and obtaining the distri- bution of a sequential two-step estimator by bootstrap, rather than the method discussed in section 2.5.3 and detailed in Newey (1984). Despite these other advantages, the statistical literature has focused on small-sample performance of the bootstrap.
The bootstrap can be applied to estimation of moments of the distribution of a statistic, testing hypotheses, and construction of confidence intervals.
We begin with use of the bootstrap to estimate standard errors. Let ˆθjdenote
the estimator of the jth component of the parameter vectorθ. The bootstrap
1. Form a new pseudosample of size n, (yl∗, x∗l), l = 1, . . . , n, by sam- pling with replacement from the original sample (yi, xi), i = 1, . . . , n.
2. Obtain the estimator, say ˆθ1with jthcomponent ˆθj,1, using the pseu-
dosample data.
3. Repeat steps 1 and 2 B times giving B estimates ˆθj,1, . . . , ˆθj,B.
4. Estimate the standard deviation of ˆθj using the usual formula for the
sample standard deviation of ˆθj,1, . . . , ˆθj,B, or
seBoot[ ˆθj]= + , , - 1 B− 1 B b=1 ( ˆθj,b− ¯θj)2 (5.55)
where ¯θj is the usual sample mean ¯θj = (1/B)
B
b=1θˆj,b. The esti-
mated standard error is the square root ofVBoot[ ˆθP, j].
The bootstrap is very easy to implement in this example, given a resampling algorithm and a way to save parameter estimates from the B simulations. The only drawback is that if estimating the model once takes a long time, estimating it B times may be too computationally burdensome. Efron and Tibshirani (1993, p. 52) state that B = 200 is generally sufficient for standard error estimation. The method is easily adapted to statistics other than an estimator – replace ˆθ by the statistic of interest, and to estimates of moments other than the sample standard deviation.
The bootstrap can additionally provide improved estimation of the distribu- tion of a statistic in small samples, in the sense that as n → ∞ the bootstrap estimator converges faster than the usual first-order asymptotic theory. These gains occur because in some cases it is possible to construct the bootstrap as a numerical method to implement an Edgeworth expansion, which is a more refined asymptotic theory than the usual first-order theory. A key requirement for improved small-sample performance of the bootstrap is that the statistic being considered is asymptotically pivotal, which means that the asymptotic distribution of the statistic does not depend on unknown parameters.
We present a version of the bootstrap for hypothesis tests that yields improved small-sample performance. Consider testing the hypothesis H0 : θj = θj 0
against H0:θj = θj 0, where estimation is by the PoissonPMLE. The t statistic
used is tj = (ˆθj − θj 0)/sj, where sj is the robust sandwich standard error
estimate for ˆθj which assumes that (yi, xi) is iid.On the basis of first-order
asymptotic theory, we would reject H0at levelα if |tj| > zα/2.
The bootstrap procedure to test H0is as follows:
1. Form a new pseudosample of size n, (yl∗, x∗l), l = 1, . . . , n, by sam-
pling with replacement from the original sample (yi, xi), i = 1, . . . , n.
2. Obtain the estimator ˆθj,1, the standard error sj,1, and the t statistic
tj,1= (ˆθj,1− θj 0)/sj,1for the pseudosample data.
3. Repeat steps 1 and 2 B times, yielding tj,1, . . . , tj,B.
4. Order these B t statistics and calculate tj,[α/2]and tj,[1−α/2], the lower
5. Reject H0at levelα if tj, the t statistic from the original sample, falls
outside the interval (tj,[α/2], tj,[1−α/2]).
For confidence intervals the same bootstrap procedure is used, except that at step 2 one forms t∗j,1= (ˆθj,1− ˆθj)/sj,1, centering around the estimate ofθj
from the original example, and at the last stage one constructs the 100(1− α)% confidence interval ( ˆθj− t∗j,[α/2]sj, ˆθj+ t∗j,[1−α/2]sj).
This bootstrap procedure leads to an improved small-sample performance in the following sense. Let α be the nominal size for a test procedure. Usual asymptotic theory produces t tests with actual size α + O(n−1/2), whereas this bootstrap produces t tests with actual size α + O(n−1). This refinement is possible because it is the t statistic, whose asymptotic distribution does not depend on unknown parameters, that is bootstrapped. For both hypothesis tests and confidence intervals the number of iterations should be larger than for standard error estimation, say B= 1000.
An alternative bootstrap method is the percentile method. This calculates θj,[α/2] and θj,[1−α/2], the lower and upper α/2 percentiles of ˆθj,1, . . . , ˆθj,B.
Then one rejects H0 : θj = θj 0 against Ha : θj = θj 0 ifθj 0 does not lie in
(θj,[α/2], θj,[1−α/2]), and one uses (θj,[α/2], θj,[1−α/2]) as the 100(1− α)% confi-
dence interval. This alternative procedure is asymptotically valid but is no better than using the usual asymptotic theory because it is based on the distribution of ˆθj, which unlike tj depends on unknown parameters. Similarly, using the
usual hypothesis tests and confidence intervals, with the one change that sj is
replaced by a bootstrap estimate, is asymptotically valid but no better than the usual first-order asymptotic methods.
These alternative approaches illustrate the need to bootstrap the right statis- tic to achieve small-sample performance gains. Theoretically inferior methods may still be very useful in actual applications, however, as they do not require computation of sj using potentially complicated asymptotic results. Also, for
very large samples there may be little need for asymptotic refinements. The bootstrap can also be used for bias correction. Consider estimation of θj, the jthcomponent ofθ. Let ˆθj denote the usual estimator ofθj using the
original sample, and let ¯θjdenote the average over B bootstrap replications of
the bootstrap estimates. The estimator ¯θjis a bootstrap measure of E[ ˆθj], so the
bootstrap estimate of bias is ( ¯θj− ˆθj). Before giving a general formula, consider
a specific example of bias correction in which ˆθj = 4 and ¯θj = 5. Then ˆθj is
upward-biased with bias of 1 because the bootstrap estimate of E[ ˆθj] is 5. To
correct for this upwards bias in the estimator ˆθjwe subtract the bias from the
sample estimate ˆθj, giving a bias-corrected estimate of 3. More generally, the
bias-corrected estimate ofθj is ˆθj− (¯θj− ˆθj)= 2ˆθj− ¯θj. Note that the bias-
corrected estimate ofθjis not ¯θj. Efron and Tibsharani (1993, p. 138) provide
several other caveats on using the bootstrap for bias correction.
A key requirement for validity of the bootstrap is that resampling be done on a quantity that isiid. The bootstrap pairs procedure ensures this, resampling jointly the pairs (yi, xi), which are assumed to beiid. In the linear model with
to bootstrap or resample the residuals. Efron and Tibshirani (1993, p. 113) discuss bootstrapping pairs, rather than residuals, for the linear model withiid
errors where both approaches are possible. For count data, bootstrapping the residuals is not valid as the errors are heteroskedastic, for example, and therefore notiid.
Horowitz (1997) gives a detailed example of bootstrap hypothesis tests for the linear model with heteroskedasticity. In addition to bootstrap pairs, he uses the wild bootstrap of Liu (1988); see also Mammen, 1993), which imposes on the bootstrap the restriction that the conditional mean of the error is zero. The wild bootstrap performs considerably better than bootstrapping pairs. These methods can be adapted to the PoissonPMLE. Presumably further gains can be obtained by imposing any additional moment assumptions that might be made, such as theGLMassumption that the variance is a multiple of the mean. For fully parametric models such as the hurdle model one can perform hypothesis tests using a parametric bootstrap.
For time series data, dependence is a potential problem. In the linear model it is accounted for by assuming an autoregressive moving average error structure and resampling the underlying white noise error, or by using the moving-blocks bootstrap in which blocks are independent but the correlation structure within blocks is preserved. These time series methods are in their infancy.
5.5.2 Other Corrections
Small-sample corrections to testing in count data models have rarely been done, although this should change rapidly as the bootstrap becomes increasingly used. To date the leading example of small-sample correction in count models has been toLMtests for overdispersion, using an approach due to Dean and Law- less (1989a), which differs from the Edgeworth expansion and bootstrap. This method can be applied to anyGLM, not just the Poisson.
Dean and Lawless (1989a) considered theLMtest statistic for Poisson against
NB2given in (5.42). The starting point is the result in McCullagh and Nelder (1983, appendix C) that forGLMdensity with meanµi and varianceV[yi], the
residual (yi− ˆµi) has approximate variance (1− hii)V[yi], where hiiis the ith
diagonal entry of the hat matrix H defined in (5.12). Applying this result to the Poisson, it follows that
E[(yi− ˆµi)2− yi] (1 − ˆhii) ˆµi− ˆµi −hiiµˆi. (5.56)
This leads to small-sample bias under H0 : E[(yi − µi)2− yi], which can be
corrected by adding ˆhiiµˆito components of the sum in the numerator of (5.42),
yielding the adjustedLMtest statistic
TaLM= n i=1 1 2µˆ −2 i g2( ˆµi) −1/2 × n i=1 1 2µˆ −2 i g( ˆµi){(yi− ˆµi)2− yi+ ˆhiiµˆi}. (5.57)
For the Poisson with exponential mean function, ˆhiiis the ithdiagonal entry in
W1/2X(XWX)−1XW1/2where W=Diag[ ˆµi] and X is the matrix of regres-
sors.
Dean and Lawless (1989a) considered tests of Poisson againstNB2overdis- persion, g(µi)= µ2i. The method has also been applied to otherGLMmodels.
Application to overdispersion in the binomial model is relatively straightfor- ward and is presented in Dean (1992). Application to a truncated Poisson model, also aGLM, is considerably more complex and is given by Gurmu and Trivedi (1992). For data left-truncated at r , meaning only yi ≥ r is observed, the
adjustedLMtest for Poisson against negative binomial is
TaLM= [ ˆIαα]−1/2 n i=1 1 2µˆ −2 i g( ˆµi) × {(yi− ˆµi)2− yi+ (2yi− ˆµi− r + 1)λ(r − 1, ˆµi) ˆµi},
where ˆIααis the scalar subcomponent forα of the inverse of the information matrix−E[∂2L/∂θ∂θ] evaluated at ˆθ = ( ˆβ, 0), see Gurmu and Trivedi (1992,
p. 350), andλ(r − 1, µ) = f (y, µ)/1 − F(y, µ) where f (·) and F(·) are the untruncated Poisson density andcdf.
The procedure has a certain asymmetry in that a small-sample correction is made only to the term in the numerator of the score test statistic. Conniffe (1990) additionally considered correction to the denominator term.
This method for small-sample correction of heteroskedasticity tests is much simpler than using the Edgeworth expansion, which from Honda (1988) is surprisingly complex even for the linear regression model under normality. The method cannot be adapted to tests of the regression coefficients themselves, however, as the score test in this case involves a weighted sum of (yi− ˆµi) and
the above method yields a zero asymptotic bias for (yi − ˆµi). Small-sample
adjustments are most easily done using the bootstrap, which as already noted is actually an empirical implementation of an Edgeworth expansion.