Theorem.: T T H + rr /r r ; r
, crnr; ^ - 1 crra*
~_1Z ^ W X y - n) 'V A J / 'V AXTAA* __ -1 Proof: H = W ^ X WX) X I W^[X|z] T A T A , X WX X Wz -1 pX 1Ti n"»/\. 'T'/v T _z*WX z Wz z (II.1) Let A (II.1) XTWX, B = XTWz, D can be written as:H = W^[X|z] p -1 rA +FE F- I T -1 T -E F 1
'T'/v
z Wz. By the matrix inversion result,
• Ti X_ T z (II.2) where 'p -1 »y*/N rp/s E = D - B A B = z Wz - z WX(X WX) X Wz 1 TA 1 pA
is a scalar and F = A B = (X WX) X Wz is a pxl vector. Multiplying (II.2) out, we obtain:
~ U7^ fvr A_ 1 1 T7T7-lirT1v T T ^ 1 T
H = W (X[A +FE F JX - z[E F JX - X[FE Jz + zE z }W Recall H = W'ScA *X^W^ and z = Xß + W *(y-p) so that
(I-H)W^z = (I-H)W^[XP + W _1(y-|l)] = (I-H)W",/4(y-4)
= (I-H)r = r
as (I-H) is the projection matrix spanning the residual (r) space. Now since E = [(I-H)W^z]T [(I-H)wV | = rT r , consequently, H = H + (rT r)_1W^{XFFTXT - zFTXT - XFzT + zzT }W^ = H + (rT r )~ 1W ^ (z-XF)(z-XF)TW^ . But W^(z-XF) = (I-K)W^z = r , hence ~ T -1 T H = H + ( r r ) rr .
REFERENCES
AKAHIRA, M. & TAKEUCHI, K. (1981). Asymptotic Efficiency of Statistical Estimators. New York : Springer-Verlag.
-2
AMEMIYA, T. (1980). The n -order mean squared errors of the maximum likelihood and the minimum logit Chi-square estimator. Ann. Statist. 8, 488-505.
ANDREWS, D.F. & PREGIBON, D. (1978). Finding the outliers that matter. J. Royal Statist. Soc. B 40, 85-93.
ATKINSON, A.C. (1981). Two graphical displays for outlying and influential observations in regression. Biometrika 68, 13-20.
ATKINSON, A.C. (1982). Regression diagnostics, transformations and constructed variables. J. Royal Statist. Soc. B 44, 1-36.
ATKINSON, A.C. (1985). Plots, Transformations, and Regression. Oxford : University Press.
BAKER, R.J. & NELDER, J.A. (1978). The GLIM system manual, Release 3. Oxford : Numerical Algorithms Group Ltd.
BELSLEY, D.A., KUH, E . , & WELSCH, R.E. (1980). Regression Diagnostics. New York : John Wiley.
BOX, G.E.P. & TIAO, G.C. (1973). Bayesian Inference in Statistical Analysis. New York : Addison-Wesley.
BOX, M.J. (1971). Bias in nonlinear estimation. J. Royal Statist. Soc. B 33, 171-201.
BRADLEY, R.A. & SRIVASTAVA, S.S. (1979). Correlation in polynomial regression. Amer. Statist. 33, 11-14.
CHATTERJEE, S. & HADI, A.S. (1986). Influential observations, high leverage points, and outliers in linear regression. Statistical Science 1, 379-416.
CLEVELAND, W.S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74, 829-836.
COOK, R.D. 8l WANG, P.C. (1983). Transformations and influential cases in regression. Technometrics 25, 337-343.
COOK, R.D. & WEISBERG, S. (1980). Characterizations of an empirical influence function for detecting influential cases in regression. Technometrics 22, 495-508.
COOK, R.D. & WEISBERG, S. (1982). Residuals and Influence in Regression. New York : Chapman and Hall.
COPAS, J.B. (1983). Regression, prediction and shrinkage. J. Royal Statist. Soc. B 45, 311-354.
Hall.
DRAPER, N.R. & JOHN, J.A. (1981). Influential observations and outliers in regression. Technometrics 23, 21-26.
DRAPER, N.R. & VAN NOSTRAND, R.C. (1979). Ridge regression and James-Stein estimation: review and comments. Technometrics 21, 451-466.
EZEKIEL, M. (1924). A method of handling curvilinear correlation for any number of variables. J. Amer. Statist. Assoc. 19, 431-453.
FAHRMEIR, L. & KAUFMANN, H. (1985). Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models. Ann. Statist. 13, 342-368.
FAREBROTHER, R.W. (1983). An examination of recent criticisms of ridge regression simulation designs. Commun. in Statist. A 12, 2549-2555. FOWLKES, E.B. (1987). Some diagnostics for binary logistic regression
via smoothing. Biometrika 74, 503-515.
GIBBONS, D.G. (1981). A simulation study of some ridge estimators. J. Amer. Statist. Assoc. 76, 131-139.
GOLDSTEIN, M. (1976). Bayesian analysis of regression problems. Biometrika 63, 51-58.
GOLUB, G.H., HEATH, M. & WAHBA, G. (1979). Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215-223.
GRAY, J.B. & LING, R.F. (1984). K-Clustering as a detection tool for influential subsets in regression. Technometrics 26, 305-330.
GUNST, R.F. (1983). Regression analysis with multicol1inear predictor variables: definition, detection, and effects. Commun. in Statist. A
12, 2217-2260.
HOAGLIN, A.E. & WELSCH, R.E. (1978). The hat matrix in regression and anova. The Amer. Statist. 32, 17-22.
HOERL, A.E. & KENNARD, R.W. (1970a). Ridge regression: biased estimator for non-orthogonal problems. Technometrics 12, 55-67. HOERL, A.E. & KENNARD, R.W. (1970b). Ridge regression: applications to
non-orthogonal problems. Technometrics 12, 69-72.
HOERL, A.E., KENNARD, R.W., & BALDWIN, K.F. (1975). Ridge regression: some simulations. Commun. in Statist. 4, 105-123.
HOERL, R.W., SCHUENEMEYER, J.H., & HOERL, A.E. (1986). A simulation of biased estimation and subset selection regression techniques. Technometrics 28, 369-380.
JENNINGS, D.E. (1986). Outliers and residual distributions in logistic regression. J. Amer. Statist. Assoc. 81, 987-990.
variables in the logistic regression model for binary data. Biometriha 74, 495-501.
LANDWEHR, J. (1986). Using residual plots to detect nonlinearity in multiple regression. AT & T Bell Lab Technical Report #15.
LANDWEHR, J . , PREGIBON, D. , & SHOEMAKER, A. (1984). Graphical methods for assessing logistic regression models. J. Amer. Statist. Assoc. 79, 61-71.
LARSEN, W.A. & MCCLEARY, S.A. (1972). The use of partial residual plots in regression analysis. Technometrics 14, 781-790.
LAWLESS, J.F. & WANG, P. (1976). A simulation study of ridge and other estimators. Commun. in Statist. 2, 105-123.
LEAMER, E.E. (1978). Specification Searches. New York : John Wiley. LEE, A.H. (1987a). Diagnostic displays for assessing leverage and
influence in generalized linear models. Aust. J. Statist. 29, in press.
LEE, A.H. (1987b). Assessing partial influence in generalized linear models. Biometrics 43, to appear.
LEE, A.H. & SILVAPULLE, M.J. (1987). Ridge estimation in logistic regression. Accepted for publication in Commun. in Statist. B subject to minor amendments.
LEE, T.S. & CAMPBELL, D.B. (1985). Selecting the optimum k in ridge regression. Commun. in Statist. A 14, 1589-1604.
LEHMANN, E.L. (1983). Theory of Point Estimation. New York : John Wiley.
LINDLEY, D.V. & SMITH, A.F.M. (1972). Bayes estimates for the linear model. J. Royal Statist. Soc. B 34, 1-18.
MANSFIELD, E.R. & CONERLY, M.D. (1987). Diagnostic value of residual and partial residual plots. Amer. Statist. 41, 107-116.
MANSFIELD, E.R. & HELMS, B.P. (1982). Detecting multicollinearity. Amer. Statist. 36, 158-160.
MARQUARDT, D.W. (1970). Generalized inverses, ridge regression, biased linear estimation and nonlinear estimation. Technometrics 12, 591-612.
MASON, R.L. 8i GUNST, R.F. (1985). Outlier-induced col linearities. Technometrics 27,401-407.
MCCULLAGH, P. (1986). On the conditional distribution of goodness-of- fit statistics for discrete data. J. Amer. Statist. Assoc. 81, 104-107.
MCCULLAGH, P. & NELDER, J.A. (1983). Generalized Linear Models. London : Chapman and Hall.
some ridge type estimators. J. Amer. Statist. Assoc. 70, 407-416. MILLER, T.I. &. TRACY, R.L. (1984). Further results concerning ridge
regression. Commun. in Statist. A 13, 251-262.
MOSTELLER, F. & TUKEY, J.W. (1977). Data Analysis and Regression. Reading, Mass. : Addison-Wesley.
NELDER, J.A. & WEDDERBURN, R.W.M. (1972). Generalized linear models. J. Royal Statist. Soc. A 135, 370-384.
NEWHOUSE, J.P. & OMAN, S.D. (1971). An evaluation of ridge estimators. Rand Corp. report R-716-PR, 1-28.
NORDBERG, L. (1982). A procedure for determination of a good ridge parameter in linear regression. Commun. in Statist. B 11, 285-309. PAGEL, M.D. (1981). Comment on Hoerl and Kennard’s ridge regression
simulation methodology. Commun. in Statist. A 10, 2361-2367.
PIERCE, D.A. & SCHAFER, D.W. (1986). Residuals in generalized linear models. J. Amer. Statist. Assoc. 81, 977-986.
PREGIBON, D. (1979). Data Analytic Methods for Generalized Linear Models. Unpublished Ph.D. thesis : University of Toronto.
PREGIBON, D. (1980). Goodness of link tests for generalized linear models. Appl. Statist. 29, 15-24.
PREGIBON, D. (1981). Logistic regression diagnostics. Ann. Statist. 9, 705-724.
PRESS, S.J. & WILSON, S. (1978). Choosing between logistic regression and discriminant analysis. J. Amer. Statist. Assoc. 73, 699-705. SCHAEFER, R.L. (1983). Bias correction in maximum likelihood logistic
regression. Statist, in Medicine 2, 71-78.
SCHAEFER, R.L. (1986). Alternative estimators in logistic regression when the data are collinear. J. Statist. Comp. Simul. 25, 75-91. SCHAEFER, R.L., ROI, L.D., & WOLFE, R. A. (1984). A ridge logistic
estimator. Commun. in Statist. A 13, 99-113.
SILVAPULLE, M.J. (1981). On the existence of maximum likelihood estimators for the binomial response models. J. Royal Statist. Soc. B 43, 310-313.
SILVAPULLE, M.J. (1985). An examination of criticisms of ridge regression simulation methodology. Commun. in Statist. B 14, 829-835.
STEWART, G.W. (1987). Collinearity and least squares regression. Statistical Science 2, 68-100.
THOMPSON, R. & BAKER, R.J. (1981). Composite link functions in generalized linear models. Appl. Statist. 30, 125-131.
Technometrics 27, 273-276.
WEDDERBURN, R.W.M. (1976). On the existence and uniqueness of maximum likelihood estimates for certain generalized linear models. Biometriha 63, 27-32.
WEISBERG, S. (1985). Applied Linear Regression (2nd edition). New York : John Wiley.
WICHERN, D.W. & CHURCHILL, G.A. (1978). A comparison of ridge estimators. Technometrics 20, 301-311.
WILLIAMS, D.A. (1982). Extra-binomial variation in logistic linear models. Appl. Statist. 31, 144-148.
WILLIAMS, D.A. (1983). The use of the deviance to test the goodness of fit of a logistic linear model to binary data. GLIM Newsletter 6, 60-63.
WILLIAMS, D.A. (1984). Residuals in generalized linear models. Proceedings of the XIIth International Biometric Conference, Tokyo, 59-68.
WILLIAMS, D.A. (1987). Generalized linear model diagnostics using the deviance and single case deletions. Appl. Statist. 36, 181-191. WOOD, F.S. (1973). The use of individual effects and residuals in