On the bootstrap and confidence intervals

On the Bootstrap and

Confidence Intervals

by

Michael A. Martin

A thesis submitted for the degree of Doctor of Philosophy of the

Australian National University

C hapter One of this thesis reviews work published by others in order to provide a

background and motivation for the work that follows. Unless otherwise stated in

the text, the remaining chapters describe my own work, supervised by Professor

P. G. Hall and published jointly with him.

I could n o t have h o p ed to com plete th is work if it were n o t for th e su p p o rt, assistan ce, a n d frien d sh ip of a g reat m an y people.

F irstly , I th a n k P rofessor P e te r H all for his th o u g h tfu l an d excellent su p er­ vision. I have b en efited g re a tly from P e te r ’s insig h t, a n d his friendship, and could no t have w ished for a b e tte r supervisor. I also w ish to th a n k Professor W illiam Schucany, w ho v isited th e A u stra lia n N atio n al U niversity from S o u th ­ e rn M eth o d ist U niversity, D allas, Texas, d u rin g th e course of th is work, for his frien d ly advice an d several helpful discussions. I am g ratefu l as well to th e staff in b o th D e p a rtm e n ts of S ta tistic s a t A .N .U . for th e ir su p p o rt a n d assistance, p a r­ tic u la rly Professors C hris H eyde a n d C hip H eath co te a n d D octors Des Nicholls a n d A lan W elsh. I have been very fo rtu n a te to have w orked in an extrem ely frien d ly an d stim u la tin g en v iro n m en t, w hich h as m ad e m y tim e a t A.N.U. m ost enjoyable.

A special w ord of th a n k s goes to M rs A nn M illigan, w ho ex p ertly an d su­ p e rb ly tackled th e difficult ta sk of ren d erin g m y thesis in to w ord-processed form . I am also very g ratefu l to B a rb a ra La Scala an d R ussell S ta n d ish for th e ir help w ith th e ed itin g a n d p rin tin g of th e thesis.

I am especially in d e b te d to m y fam ily an d friends, w ho have su p p o rte d a n d en co u rag ed m e over th e last few years; to M um a n d D ad , a n d m y d ear g ra n d m o th e r, Nell, for th e ir endless su p p ly of love a n d encouragem ent; an d to th e m an y special people, to o num ero u s to m en tio n sep arately , w ho have talked w ith me, listen ed to m e, a n d above all, accepted m e as a friend.

T h e following p a p ers have been su b m itte d for p u b lic a tio n from th e w ork in this thesis:

Hall, P. a n d M artin , M .A. (1988a) O n th e b o o ts tra p an d tw o-sam ple problem s.

Austral. J. S ta tist. 3 0 A , 179-192.

H all, P. an d M a rtin , M .A. (1988b) O n b o o ts tra p resam p lin g a n d ite ra tio n .

B io m e trik a, in press.

H all, P. a n d M artin , M .A. (1988c) E x act convergence ra te of b o o ts tra p q u a n ­ tile variance e stim a to r. Probab. Theor. and Rel. Fields, in press.

H all, P. a n d M artin , M .A. (1988d) A n o te on th e accu racy of b o o ts tra p p e r­ centile m e th o d confidence in terv als for a qu an tile. S ta tist. Probab. L e tte rs, in press.

Hall, P. a n d M artin , M .A . (1988e) O n th e e rro r in c u rre d using th e b o o ts tra p variance e stim a te w hen co n stru ctin g confidence intervals for q u an tiles. J. Roy. S ta tist. Soc. Ser. B, s u b m itte d , S eptem b er, 1988.

In this thesis, we discuss th e use of b o o ts tra p m e th o d s for co n stru ctin g con­ fidence intervals in a w ide variety of circum stances. In p a rtic u la r, we provide a sy m p to tic th eo ry for b o o ts tra p confidence intervals b ased on E d g ew o rth ex p an ­ sions in several pro b lem s, a n d develop a general b o o ts tra p resam pling scheme ap p licab le to a large ran g e of sta tistic a l problem s. T h e thesis is divided into five ch ap ters. T h e first gives a b rief in tro d u c tio n to b o o ts tra p m eth o d s a n d Edge- w o rth ex p an sio n th eo ry to provide a b ack g ro u n d a n d m o tiv atio n for o u r la te r work.

T h e second c h a p te r p resen ts a sy m p to tic th eo ry for th e b o o ts tra p in two- sam ple problem s. If th e sam ples are of sizes m a n d n, th e n o u r resu lts show th a t one- an d tw o-sided percentile-2 confidence intervals for a difference betw een two m ean s have coverage e rro r 0 ( m _1 + n - 1 ), a n d th a t sy m m etric tw o-sided intervals have coverage e rro r 0 ( m -2 -\-n~2). F u rth erm o re , w hen th e p o p u latio n s are N orm al, coverage e rro r of all percentile-2 intervals dro p s to 0 ( m~ 2 -fi n - 2 ). O u r a sy m p to tic analysis, a n d th e resu lts of a sim u latio n study, in d icate th a t percentile-2 is a re sp ec ta b le solu tio n to th e B eh ren s-F ish er problem : th a t of te stin g for th e difference betw een m eans of two N orm al p o p u la tio n s, w ith o u t m aking a ssu m p tio n s a b o u t variances.

A c en tral c o n trib u tio n of o u r thesis is th e in tro d u c tio n in C h a p te r 3 of a gen­ eral resam p lin g p rin cip le w hich encom passes b o o ts tra p resam p lin g an d b o o ts tra p ite ra tio n . O u r prin cip le is ap p licab le to a wide ran g e of s ta tistic a l problem s, such as th e co n stru ctio n of confidence intervals a n d h y p o th esis te sts, a n d bias red u c­ tio n for p o in t e stim atio n . T h e p rin cip le is illu s tra te d th ro u g h several exam ples, including bias co rrectio n a n d L 1 shrinkage.

two-p o sitio n of critical two-p o in t an d m ean interval length, an d illu stra te th e a two-p two-p lic atio n of b o o ts tra p coverage correction to B a rtle tt-c o rre c te d likelih o o d -ratio te sts and confidence intervals for co rrelatio n coefficients.

Title page

Preface

Acknowledgements

Related publications

Abstract

Contents

Chapter One

The Bootstrap, Edgeworth Expansions and Confidence Intervals 1

1. Introduction

1

2. The bootstrap and confidence intervals

3

3. Principles of Edgeworth expansion

6

3.1. Basic results

6

3.2. Sums o f independent random variables

8

3.3. More general statistics

12

3.4. A model fo r valid Edgeworth expansions

16

3 5 . Comish-Fisher expansions

20

3.6. Bibliographical notes

21

4. Summary of thesis

23

Chapter Two

On Bootstrap Confidence Intervals and Two-sample Problems

25

1. Introduction

25

2. Construction of bootstrap confidence intervals

26

3. Comish-Fisher expansions of bootstrap critical points

28

4. Second-order correctness

31

5. Accelerated bias correction

32

6. Lengths of two-sided, equal-tailed intervals

33

7. Coverage error

34

8. Symmetric confidence intervals

36

9. Short confidence intervals

37

Appendix A2.1: Proof o f Lemma 2.3 49 Appendix A2.2: Derivation o f constants u and v in the percentile-t case 53 Appendix A2.3: Derivation o f the polynomial q31 in the Cornish-Fisher 55

expansion o f percentile-t critical point

Chapter Three

Bootstrap Resampling and Iteration

56

1. Introduction 56

2. The principle 57

2.1. General formulation o f statistical problems 57

2 2 . Resampling solutions to statistical problems 59

3. Iterating the principle 60

4. Bias correction 65

5. Lehmann-type confidence intervals 66

6. L^shrinkage 71

7. Proofs 71

7.1. P roof o f Theorem 3.1 71

7 2 . P roof o f Lemma 3.2 76

7 3 . Asymptotics fo r Lehmann-type intervals: p ro o f o f

Theorem 3.3 77

7.4. Asymptotics fo r L1 2-shrinkage 80

Appendix A3.1: P roof o f Lemma A3.1 82

Chapter Four

On Bootstrap Iteration for Coverage Correction in Confidence

Intervals

83

1. Introduction, definitions and notation 83

1.1. Introduction 83

1 2 . Parametric and nonparametric bootstraps 84

1.3. Definitions and notation 85

2.2. One-sided intervals

87

2 3 . Two-sided intervals

88

2.4. Coverage correction and the parametric bootstrap

90

3. Effect of bootstrap coverage correction on critical point

91

3.1. One-sided intervals

91

3 2 . Two-sided intervals

98

4. Effect of bootstrap coverage correction on interval length

101

5. Simulation study

104

6. Applications of coverage correction

104

6.1. Bootstrap coverage correction for Bartlett-corrected

likelihood-ratio tests

105

62. Better nonparametric bootstrap confidence intervals

for the correlation coefficient

107

7. Proofs

116

7.1. Proof o f Theorem 4.1

116

72. Proof o f Theorem 4.2

118

73. Proof o f Theorem 4.3

123

7.4. Proof o f Theorem 4.4

125

73 . Proof o f Theorem 4 3

126

7.6. Proof o f Theorem 4.6

127

7.7. Asymptotics for Bartlett correction: proof that

terms o f order m512 vanish in (4.49)

127

Appendix A4.1: Proof o f Lemma A4.1

134

Chapter Five

Bootstrap Methods for Constructing Confidence Intervals for

Quantiles

136

1. Introduction

136

2. On the accuracy of bootstrap percentile method confidence

intervals for a quantile

138

3. The exact convergence rate of the bootstrap quantile variance

estimator

141

4. The error incurred using the bootstrap variance estimator

6. Proofs 150

6.1. Proof o f Theorem 5.1

150

62. Proof o f Theorem 5.2

151

6.3. Proof o f Theorem 5.3

160

6.4. Proof o f Theorem 5.4

171

63. Proof o f Theorem 5.5

171

Appendix A5.1: Proof o f (5.3)

173

som etim es

.'

"Aft!”, said% a66it, zuho never let things come to him, 6 u t aCzvays

zvent a n d fe tc h e d them.

C H A P T E R 1

T H E B O O T S T R A P , E D G E W O R T H E X P A N S IO N S

A N D C O N F ID E N C E IN T ER V A LS

1. IN T R O D U C T IO N

In recen t years, vast im provem ents in th e speed a n d cost of c o m p u ta tio n have re s u lte d in th e developm ent of a m u ltitu d e of new sta tistic a l theories an d tech n iq u es w hich in th e p a st w ould have been dism issed as u n th in k a b ly slow a n d expensive to use. Even so, m an y of to d a y ’s m o st com m only-used sta tistic a l m e th o d s w ere developed in th e first few decades of th e 20’th century, w hen m o st c a lc u la tio n s h a d to b e done lab o rio u sly by h a n d , or w ith th e help of a m ech an ical desk calcu lato r. As a resu lt, th ese older m e th o d s often relied u p o n several sim p lify in g a ssu m p tio n s w hich m ad e th e m m a th e m atica lly tra c ta b le .

C e n tra l to th e developm ent of m an y of these tech n iq u es is th e assu m p tio n th a t th e d a ta conform to a N o rm al or G au ssian d istrib u tio n . Indeed, th e N orm al d is trib u tio n plays a d o m in a tin g role in m uch of classical s ta tistic a l theory. T h e reliance of th e se older m e th o d s on a ssu m p tio n s such as th a t of N o rm ality places severe lim ita tio n s on th e ir w ide applicability, a n d raises th e serious q u estio n of w h at to do w h en th e a ssu m p tio n s are n o t satisfied. C o m p u ter-in ten siv e m eth o d s such as E fro n ’s (1979a) b o o ts tra p offer us freedom from th ese lim itin g a ssu m p ­ tions by re p la cin g s ta n d a rd m o d el-b ased a ssu m p tio n s w ith an en o rm o u s am o u n t of calcu latio n . T h e b o o ts tra p g en erates a m odel for th e p a re n t p o p u la tio n di­ rectly fro m th e d a ta , a n d so reflects all th e c h aracteristics of th e sam ple.

0.95, th e tru e m ean lies w ith in 1.96a-/ s j n of th e sam ple m ean, say X . O f course, o u r exact confidence s ta te m e n t relies heavily on o u r a ssu m p tio n s of N o rm ality a n d know n variance. However, we m ay relax th ese a ssu m p tio n s a n d still maJke a p p ro x im a te confidence s ta te m e n ts like th e above. Suppose now th a t th e v ari­ ance o'2 is unknow n, a n d we e stim ate it by th e sam ple variance, say a 2. T h en , w ith p ro b a b ility a p p ro x im a te ly 0.95, th e tru e m ean lies w ith in 1.96<j/>/n of th e sam ple m ean, th e e rro r in a p p ro x im a tio n arising from th e difference betw een dis­ trib u tio n fu nctions of N orm al a n d “S tu d e n t’s” t w ith n — 1 degrees of freedom . We can also rid ourselves of th e a ssu m p tio n th a t th e d a ta are N o rm al by using th e C en tral L im it T h eo rem . M erely assum ing th a t th e d a ta are in d e p e n d e n t an d id en tically d is trib u te d o bservations from a d is trib u tio n hav in g finite varian ce, we have, w ith p ro b a b ility a p p ro x im ately 0.95 in large sam ples, th a t th e tru e m ean lies in th e ran g e X — 1.96<j/>/n to X + 1.9 6 d -/\/m T h is sta te m e n t is b ased on th e fact th a t, as n —► oo, th e ra tio n ^ ( X —/z)/<t converges in d is trib u tio n to a S ta n d a rd N orm al iV (0 ,1) d istrib u tio n .

It m ig h t seem from o u r discussion in th e last p a ra g ra p h th a t we h ave solved o u r sim ple p ro b lem w ith o u t to o m uch tro u b le, b u t a n u m b e r of q u estions rem ain to be answ ered. F irstly , confidence sta te m e n ts like th e la st are n o t e x ac t, relying on a N orm al a p p ro x im a tio n to th e d is trib u tio n of n * ( X — T w o q u estions w hich im m ed iately sp rin g to m in d are: how good is th a t N orm al ap p ro x im a tio n , p a rtic u la rly in sm all sam ples, an d can we do b e tte r by using a different m eth o d ? If it is possible to do b e tte r, how close can we get to an exact so lu tio n ? A n o th e r im p o rta n t issue is: w h a t if, in ste a d of co n stru ctin g a confidence in te rv a l for a sim ple m ean , we axe in te re ste d in finding an in terv al for a m ore com plex sta tistic , such as a co rrelatio n coefficient, o r a q u an tile of an unk n o w n d is trib u tio n ? In such cases, it m ay be difficult to find a tra n sfo rm a tio n of th e s ta tis tic of in te re st to one having an a p p ro x im a te N o rm al d istrib u tio n , a n d even if such a tra n s fo rm a tio n is available, it m ay n o t b e useful for all b u t very large sam ple sizes.

m an y of th e s ta n d a r d assu m p tio n s w hich d o m in ate classical sta tistic s.

In th is th esis, we are largely concerned w ith th e pro b lem of using th e b o o t­ s tra p to c o n stru c t confidence intervals in a wide variety of circu m stan ces. In p a rtic u la r, o u r p rim a ry aim is to co n stru ct confidence intervals w hose cover­ age p ro b a b ility ( th a t is, th e p ro b a b ility th a t th e tru e p a ra m e te r lies w ith in th e in terv al) is close to som e s ta te d level. We provide theory, b ased on E d g ew o rth ex­ p an sio n , w hich enables us to com pare th eo retically th e a sy m p to tic p erfo rm an ce of several different intervals for a p a ra m e te r, an d we reinforce o u r th eo ry w ith several sim u la tio n stu d ies w hich allow us to d eterm in e w h e th er th e resu lts of o u r a sy m p to tic analyses are reflected in practice. We also develop a general re­ sam p lin g p rin c ip le w hich guides th e o p e ra tio n of b o o ts tra p resam p lin g in a wide ran g e of p ro b lem s, a n d leads n a tu ra lly to ite ra te d b o o ts tra p schem es w hich can be u sed to im p ro v e various asp ects of a so lu tio n to a s ta tistic a l problem .

In Section 2 of th is c h a p te r we give a b rief in tro d u c tio n to b o o ts tra p th e o ry a n d its use in c o n stru c tin g confidence intervals. Section 3 discusses th e principles of E d g e w o rth ex p an sio n , a to o l w hich is used th ro u g h o u t th e thesis. Finally, in Section 4 we p ro v id e a su m m a ry of th e topics stu d ie d in th e re m a in d e r of th is thesis.

2. T H E B O O T S T R A P A N D C O N F ID E N C E IN T E R V A L S

T h e id e a b e h in d th e b o o ts tra p is a sim ple one: if you w ish to e stim a te a fu n c tio n a l of a p o p u la tio n d is trib u tio n fu n ctio n F, for in sta n ce a p o p u la tio n m ean

one w ay to p ro ceed is possibly to use th e sam e fu n ctio n al of th e sam p le d is tri­ b u tio n fu n c tio n F , in o u r ex am p le equal to th e sam ple m ean

Such an a rg u m e n t does n o t w ork in all cases; e stim atio n of a p ro b a b ility d en sity fu n c tio n is a case in p o in t. N evertheless, th e b o o ts tra p p rin cip le is ap p licab le in a w ide ra n g e o f p ro b lem s, a n d has received m uch a tte n tio n in th e s ta tistic a l lite ra tu re .

largely ex isted in descriptive an d d isp a ra te form , ra th e r th a n p re sen te d as a u n i­ fied, g en eral theory. A m a jo r featu re of E fro n 's tre a tm e n t of th e b o o ts tra p was its d ep en d e n c e o n th e co m p u ter. E fron p o in te d o u t th a t in m an y com plex situ ­ a tio n s, w here b o o ts tra p q u a n titie s are aw kw ard (if no t im possible) to com pute algebraically, th e y m ay be c o m p u ted by using M onte C arlo m eth o d s to n u m e ri­ cally e stim a te fu n ctio n als of th e em piric d is trib u tio n fun ctio n . M ore specifically, sam e-size sam p les (called “resam p les” ) m ay be d raw n w ith rep lacem en t from th e o rig in al sam p le a large n u m b e r of tim es, th e value of th e s ta tis tic of in te re st co m p u te d for each of th e resam ples, a n d th e b o o ts tra p q u a n tity co m p u te d by averaging th ese values over all resam ples. T his a p p ro a ch works because, in th e m a jo rity of ap p lic atio n s of th e b o o stra p , b o o ts tra p q u a n titie s m ay b e expressed as e x p e c ta tio n s co n d itio n al on th e sam ple. O u r developm ent in C h a p te r 3 of a g en eral re sam p lin g p rin cip le takes ad v an tag e of th is p ro p erty .

R eferences to w ork along lines sim ilar to E fro n ’s a ro u n d th e tim e of his sem inal p a p e r in clu d e B a rn a rd (1963), H ope (1968) a n d M arrio t (1979) w ho ad ­ v o c ate d th e use of M onte C arlo m eth o d s for h y p o th esis testin g ; H a rtig a n (1969,

1971, 1975), w ho u sed resam p led su b sam p les to c o n stru ct p o in t a n d in te rv a l esti­ m ates, a n d w ho stressed th e connections w ith th e jackknife of Q uenouille (1949, 1956) a n d T ukey (1958); a n d M aritz an d J a r r e t t (1978), w ho derived b o o ts tra p e stim ate s of th e variance of th e sam ple m edian. For som e highly re a d a b le expo­ sitions of b o o ts tr a p th eo ry for p o in t a n d interv al estim atio n , we refer th e read er to E fro n (1979a), D iaconis a n d E fron (1983), E fron a n d G ong (1983), E fro n and T ib sh ira n i (1986), a n d E fro n ’s (1982) m o n o g rap h . G eneral acco u n ts of b o o ts tra p th e o ry in clu d e Bickel a n d F reed m an (1981) a n d B eran (1984b).

H all (1988a) gives a d e tailed su m m a ry a n d co m p ariso n of b o o ts tra p m eth o d s for c o n stru c tin g confidence in terv als for a u n iv a ria te p a ra m e te r

0

. He identifies five m e th o d s w hich are in com m on use: tw o so-called “p e rc en tile ” m eth o d s, w hich h e te rm s “b ack w a rd s” a n d “h y b rid ” p ercen tile m e th o d s, respectively; the “p e rc e n tile -t” m eth o d ; a n d E fro n ’s b ias-co rrected a n d accelerated bias- co rrected m eth o d s. R eferences to th e tw o p ercen tile m eth o d s are n u m ero u s a n d include E fron (1979a, 1981a, 1982, 1987), a n d E fron a n d T ib sh iran i (1986), am o n g o th ­ ers. T h e “p e rc e n tile -t” m e th o d is discussed by Bickel a n d F ree d m a n (1981), H inkley a n d W ei (1984), B eran (1987a, 1987b, 1988), H all (1987a, 1987b, 1988a) a n d Singh (1987). B ias-co rrected a n d accelerated b ias-co rrected m e th o d s are in tro d u c e d a n d discussed by E fron (1981a, 1987).

schools of th o u g h t w hich d o m in ate p resen t ap p ro ach es to th e th eo ry of b o o ts tra p confidence intervals. T h e first goes rig h t back to E fro n ’s early developm ent of th e b o o ts tra p . E fro n ’s account has usually ad h ered closely to o th e r resam p lin g m eth o d s, especially th e jackknife, in its early stages, an d th e p ercen tile m eth o d s are cen tral to his ap p ro ach . T h e “back w ard s” p ercen tile m e th o d is p ro b a b ly th e m ost p o p u la r of th e techniques am ongst n o n -sta tistic ia n s, an d p e rh a p s also w ith sta tistic ia n s, an d we shall often refer to th is m e th o d as sim ply th e p e r­ centile m eth o d . H all (1988a) term s th is m e th o d th e backw ards m e th o d because it is analogous to looking u p th e w rong s ta tistic a l tab les ( th a t is, N o rm al ra th e r th a n S tu d e n t’s t tab les) backw ards ( th a t is, confusing u p p e r an d low er q u a n ­ tiles). He refers to th e o th e r p ercen tile m e th o d as th e h y b rid m e th o d b ecau se it can be in te rp re te d as looking u p th e w rong tab les th e rig h t way a ro u n d . B o th “b ack w ard s” a n d “h y b rid ” intervals are called p ercen tile-m eth o d in terv als b e­ cause th ey are b ased on th e percentiles of th e b o o ts tra p d is trib u tio n s of 9 a n d

0 — 0 , respectively, w here 9 denotes o u r e stim a te of th e p a ra m e te r 9. See H all (1988a) for m ore d etails. E fron p o in te d o u t as early as 1981 th a t th e o rd in a ry p ercen tile m e th o d s can p erfo rm very poorly. He p ro p o sed (1981a, 1987) a n a ly t­ ical corrections to p e rc en tile-m eth o d intervals b ased on tra n s fo rm a tio n theory, to im prove th e ir p erfo rm an ce, a n d he te rm e d these intervals b ia s-co rrec te d an d accelerated b ias-co rrected intervals respectively.

th o u g h t. B eran (1987a) p o in ts o u t th e close connection betw een his form of b o o ts tra p ite ra tio n (w hich he calls “p re p iv o tin g '’) an d S tu d en tizin g to o b ta in an a p p ro x im ately piv o tal s ta tistic . O th e r accounts of b o o ts tra p ite ra tio n are given by H all (1986a), Loh (1987) a n d H all an d M a rtin (1988b).

B o th of th e philosophical s ta n d p o in ts th a t we have o u tlin ed have m erit, alth o u g h we favour th e second over th e first th ro u g h o u t th is thesis. O u r choice is by no m eans unequivocal, how ever, a n d is b ased on a n u m b e r of philosophical preferences. F irstly , it seems to us m ore n a tu ra l to base confidence intervals an d hy p o th esis te sts on p iv o tal ra th e r th a n n o n p iv o tal q u a n titie s. In H all’s (1988a) term inology, we prefer to look up th e rig h t tab les to s ta r t w ith , r a th e r th a n look up th e w rong tab les a n d th e n correct for ou r erro r. T h e second is based on a view th a t co m p u ter-in ten siv e m eth o d s such as th e b o o ts tra p , w hich axe designed to avoid th e need for ted io u s an aly tic corrections, sh ould n o t have to ap p eal to such corrections. O n th e o th e r h a n d , E fro n ’s (1981a, 1987) bias corrections do possess useful invariance p ro p erties u n d e r m o n o to n e tra n sfo rm a tio n s, n o t sh ared by p ercen tile-t. It is easy to find exam ples w here each of th e p ercen tile-t an d accelerated b ias-co rrected m eth o d s fails. For exam ple, it is w ell-know n th a t th e p ercen tile-t m e th o d perfo rm s poo rly in m an y cases w here th e variance <j2 is difficult to e stim a te - see for exam ple, E fron (1981a), p.154, a n d H all, M artin an d Schucany (1988) - w hile eq u al-tailed accelerated b ias-co rrected intervals for sm all sam ples a n d h ig h n o m in al coverage p ro b a b ilities can p ro d u ce ab n o rm ally sh o rt intervals; co m pare H all (1988a).

F u rth e r n o ta tio n a n d d etails of how th e various b o o ts tra p confidence in ­ tervals are c o n stru c te d are given as we need th em . For an excellent an d lucid exposition of th e c o n stru c tio n of an d co m p ariso n betw een th e various b o o ts tra p confidence in terv als for a u n iv a ria te p a ra m e te r 0, th e re a d e r is referred to Hall (1988a). In m an y places th ro u g h o u t th is thesis we re ta in H all’s (1988a) n o ta tio n , an d use several of his resu lts in th a t p a p er.

3. T H E P R IN C I P L E S O F E D G E W O R T H E X P A N S I O N

3 .1 . B a sic r e su lts. In th is section we give a b rie f acco u n t of E d g ew o rth e x p an ­ sions as ap p ro x im a tio n s to d istrib u tio n s of e stim ate s

9

of unknow n p a ra m e te rs

9.

¥ { n * ( 0 — 9) < crx] = <£(x) + n ~ *pi(x)<f>(x)---'\-n~J^2pj(x)(p(x)-\--- , (1.1)

w here <j>{x) = (2 7r ) _ 2e _ 2x is th e S ta n d a rd N orm al den sity fu n ctio n a n d <$(x) = 0 (y ) dy is th e S ta n d a rd N orm al d istrib u tio n function. T h e fu n ctio n s pj are polynom ials w hose coefficients d ep en d on th e cu m u lan ts of 9 — 0. W e call an ex p an sio n such as (1.1) an E d g ew o rth expansion.

E q u a tio n (1.1) m ay be in v erted to show th a t th e so lu tio n x = v a of th e eq u atio n P { n ^ (# — 9) < crx] = a a d m its an expansion

v a = z a + n ~ * p u ( z a ) + n ~ 1p2i ( z a ) H--- + n ~ j / 2p j l ( z a ) -f--- , (1.2)

w here 0 < a < 1, th e pj \ ’s are polynom ials d ep en d in g on th e p j 's, a n d is th e so lu tio n of th e eq u atio n $ (2a ) = a. We te rm inverse expansions like (1.2) C o rn ish -F ish er expansions.

So far, o u r discussion has focussed on th e d is trib u tio n of th e n o n -S tu d en tize d sta tistic n ^ ( 0 — However, in m ost cases of in te re st th e a sy m p to tic v ari­ ance cr2 is unknow n, a n d is e stim a te d by a fu n ctio n a 2 of th e d a ta . In th a t circu m stan ce, we are in te re ste d in th e d is trib u tio n fu n ctio n of th e S tu d e n tiz e d s ta tistic n ^ ( 0 — 0 ) / a , w hich a d m its an expansion like (1.1) b u t w ith different polynom ials, w hich we call qj to d istin g u ish th e m from th e polynom ials pj in th e n o n -S tu d en tize d case. W e ad o p t th is n o ta tio n th ro u g h o u t th e thesis.

A n im p o rta n t ex am p le of th e types of resu lts w hich we have discussed is th a t w here 9 is a sam ple m ean , cr2 a sam ple variance, a n d 9 an d a 2 th e cor­ resp o n d in g p o p u la tio n m e an an d variance, respectively. T h is exam ple displays all of th e m a jo r fe a tu re s of m ore general E d g ew o rth expansions: pj (qj) is a polynom ial of degree a t m o st 3j — 1, a n d is an o d d o r even fu n ctio n according as j is even or o d d , respectively; a n d expansion (1.1) is valid u n d e r a p p ro p ria te m om ent an d sm o o th n ess con d itio n s on th e u n d erly in g d istrib u tio n . T h is ex am ­ ple form s th e basis for a discussion of m ore general E d g ew o rth ex p an sio n s, a n d we stu d y it in m ore d e tail in Subsection 3.2. We e x p an d o u r tre a tm e n t to m ore general expansions in S u b sectio n 3.3 a n d place o u r a rg u m e n ts on a m o re form al level. M ore d etail is given in Subsection 3.4 in th e co n tex t of a g en eral m odel u n d e r w hich form al E d g ew o rth expansions m ay be estab lish ed rigorously, an d we discuss inverse (C o rn ish -F ish e r) expansions in S ubsection 3.5.

com m on use for E d g ew o rth expansions in classical statisics is to m ake a n a ly ti­ cal co rrectio n s ra th e r sim ilar to those w hich th e b o o ts tra p m akes by n u m erical m ean s. Such ap p licatio n s of E d g ew o rth expansions are o u tsid e th e scope of th is thesis, a n d little com m ent is m ad e on th em . A lth o u g h th e th eo ry discussed in th is sectio n is q u ite general, it is n o t universal, an d th e re do exist im p o r ta n t cases w here o u r key principles are violated. A case in p o in t is th a t of th e S tu d e n tiz e d q u an tile, w hich is tre a te d separately, in C h a p te r 5.

3 .2 . S u m s o f in d e p e n d e n t r a n d o m v a r ia b le s . Let JA, X \, X2, . • • denote in d e p e n d e n t a n d id en tically d is trib u te d ra n d o m variables w ith m ean 9 = fi a n d finite v arian ce a 2. O ne e stim a te of 9 is th e sam ple m ean,

e ^ n - 'Y ^ X i ,

i=l

w hich h as varian ce n ~ l cr2. Now, S n = (9 — 9)/cr is a sy m p to tica lly N orm ally d is trib u te d w ith zero m ean an d u n it variance. C lassical p ro ced u res, such as th a t d e sc rib e d in Section 1 for th e c o n stru ctio n of a N o rm al-th eo ry confidence interval, often use th is ap p ro x im a tio n d irectly in inference a b o u t th e m ean . We are in te re ste d in th e accu racy of th is N orm al a p p ro x im a tio n , for it d eterm in es, in th e ex am p le of c o n tru c tin g a confidence in terv al for a m ean , th e coverage e rro r of th e in terv al.

W e m ay ap p ro a ch th e p ro b lem of describing erro rs in N orm al a p p ro x im a ­ tions by u sin g c h a ra c te ristic functions. Since S n is a sy m p to tica lly S ta n d a rd N or­ m al iV (0 ,1) th e n its c h a ra c te ristic fu n ctio n , t/>„, converges to th a t of a S ta n d a rd N orm al ra n d o m variable, as n —►00; th a t is

ipn (t) = E { e x p (it5 n )} —■*• e - 2< ? —00 < t < 0 0. (1.3)

Now,

= { ^ ( * / n * ) } n , C1-4 )

w here ip d en o tes th e c h a ra c te ristic fu n ctio n of Y = ( X — fi)/cr. T h e j ’th c u m u la n t

K j, of Y eq u als th e coefficient of ( j ! ) _1(zt)J in a pow er series ex p an sio n of log ip(t);

th a t is,

ip(t) = e x p { « i(it) -1- I «2( i t ) 2 4--- 1- H--- }. (1.5)

However, since

th e c u m u la n ts of Y m ay be defined in term s of th e m o m ents of Y :

= E

( Y ) ,

Ac2 = E(E'2) -

( E Y ) 2

= v ar(y ),

«3 = E ( y 3) - 3 E (y 2)E (y ) + 2

(

e y) 3

=

e

(

y

-

e y

) \

ac

4 = E ( y 4) - 4 E (y 3)E (y ) - 3 ( £ y 2)2 4- 12E

(

y2

){

e y ) 2

-

g

(

e y

)4

= E ( y - E y )4 - 3(var Y ) 2,

a n d so on; see K en d all a n d S tu a rt (1977), p.72, eq u atio n (3.43), for fo rm u lae up to acio• N o tin g th a t

E ( y )

= aci = 0 an d v a r ( y ) = «2 = 1, we have, by (1.4) and (1.5), th a t

ipn (t) = e x p{ - \ t 2 2 j,/c 3(i t )3 4---n (j 2)/2 j K j ( i t ) ] 4--- }

= e- ^ [l 4-

n ~ ^ K 3(it)3

+ n _1 { ^ /c 4(it)4 4-

^ K l ( i t ) 6}

4----] (1.6)

= e-

2

* {l 4-

n~^ri(it)

4-

n~1r2(it)

4---f

n ~ ^ 2rj{it)

4---}, (1.7)

w here rj is a poly n o m ial of degree no m ore th a n 3 j w ith real coefficients d e p en d ­ ing on th e cu m u lan ts of Y up to th e (j 4- 2) ’th , b u t n o t on n. O bserve t h a t rj is an even fu n c tio n w hen j is even, an d an o d d fu n ctio n w hen j is odd. T h is fact is a p p a re n t from in sp ec tio n of th e arg u m e n t leading to (1.7), a n d will prove vital to o u r fu tu re work. It tu rn s o u t th a t for o u r p u rp o ses we need a t m ost th e first th re e polynom ials rj in (1.7).

We now discuss inversion of th e ch arac te ristic fu n ctio n t/>„. If we rew rite (1.7) in th e form

0 n (t) = e 2* 4- n 2r\{it)e 2< 4- n 1r2(it)e 2* 4— - 4- n J//2rj ( i t ) e 2* 4---,

(

1

.

8

)

th e n since

—00

a n d e ^ = f e ltx c7$(x), expansion ( 1.8 ) suggests th e inverse ex p an sio n

P (5 n <

x)

= $ (x ) 4-

r i ~ * R \ { x )

4-

n ~ 1R,2(x)

4---4-

n ~ i f 2R j ( x )

4---,

(1.9)

w here R j ( x ) denotes a fu n c tio n w hose F o u rier-Stieltjes tra n sfo rm equals

We n ex t co m p u te th e R j 's. In te g ra tin g by p a rts re p e ate d ly in e ltx

we have

e d<&(x)

d & l \ x ) = • • • = ( —it) J

d&J\x),

from w hich it follows th a t

I ' O O

/ eitxd { ( - D ) j $ { x ) } = ( i t y e - l * ,

J —co

(

1

.

10

)

w here D is th e differential o p e ra to r d/ dx. If we in te rp re t r j ( —D) in th e obvious way as a poly n o m ial in D, th e n r j ( —D) is a differential o p e ra to r an d , by (1.10),

y o o

/ el<1 d { r j ( - D ) ^ ( x ) } = r j ( i t ) e . (1.11)

J—oo

C onsequently, from (1.11), th e fu n ctio n s R j are given by

R j ( x ) = r j ( —d / d x ) $ ( x ) . (1.12)

Now observe th a t for j > 1, ( —D )J $ ( x ) = — Hej -i (x)<f )(x), w here th e fu n ctio n s

H e j are th e H erm ite polynom ials:

H e0(x) = 1, H e i ( x ) = x , H e2( x) = x2 — 1, i7 e 3(x ) = x (x 2 — 3),

He ±( x ) = x 4 — 6 x 2 + 3, He $ ( x ) = x (x 4 — 10x2 + 15),

a n d so on; see, for exam ple, P e tro v (1975), p.137. T h ey are o rth o g o n a l w ith re­ spect to th e w eight fu n ctio n <^>, a n d axe n o rm alized so th a t th e leading coefficient is unity. N ote th a t H e j is of degree

j

a n d is an even or o d d fu n ctio n according as

j

is even or od d . As a re su lt, for general

j

> 1, R j ( x ) = pj(x)(f>(x) , w here pj

is of degree no m o re th a n 3

j

— 1 a n d is o d d for even

j

a n d even for o d d j , since

rj is of degree no m ore th a n 3

j

an d is e v e n /o d d for e v e n /o d d

j

. In p a rtic u la r, observe from (1.6) th a t

Pi(x)

= - I « 3(x2 - 1 ) , P2(x)= - x { i K 4(x 2- 3 ) + i « 2(x 4 - 1 0 i 2+ 1 5 )} . (1.13)

F o rm u la (1.9) m ay now b e w ritte n

a n d is called an E d g ew o rth expansion of th e d is trib u tio n fu n ctio n of S n . T h ird a n d fo u rth c u m u lan ts, «3 a n d ac4, are know n as skewness a n d ku rto sis respec­ tively. T h e te rm of o rd e r n ~2 in (1.14) is a co rrectio n of th e basic N orm al

a p p ro x im a tio n for th e first o rd er effect of skewness, w hile th e te rm of order n-1 co rrects for th e first o rd e r effect of k u rto sis a n d th e second o rd er effect of

skew ness.

T h e ex p an sio n (1.14) only rarely converges as an infinite series. C ram er (1928) show ed th a t if X is ab so lu tely continuous, th e n a necessary a n d sufficient c o n d itio n for convergence is th a t E{exp( ;jY~2)} < 00, w here Y = ( X — T his

co n d itio n places severe lim itatio n s on th e tails of th e d is trib u tio n of Y , failing even if X is ex p o n en tially d istrib u te d . E x p an sio n (1.14) is usu ally in te rp re te d as an asymptoti c expansion, m ean in g th a t if th e series is te rm in a te d after a given n u m b e r of te rm s th e n th e re m a in d er is of sm aller o rd er th a n th e la st included term ; t h a t is,

P ( 5 n < x ) = $ ( x ) + n ~ *p\{x)<j>{x) + • • • 4- n ~ ^ 2pj(x)<f>(x) + o ( n ~ ^ 2). (1.15)

B h a tta c h a ry a a n d R ao (1976), T h eo rem 20.1, p.208 prove th a t sufficient re g u la r­ ity con d itio n s for (1.15) to hold, w ith th e re m a in d er of th e s ta te d o rd er uniform ly in all x, are E ( |X |J'+2) < 00 an d

lim s u p |V>(£)| < 1.

l*H°o

T h e second re stric tio n , o ften called C ra m e r’s condition, holds if th e d is trib u tio n of X h as a n o n d eg en erate ab so lu tely continuous com p o n en t - in p a rtic u la r, if

3 .3 . M o r e g e n e r a l s t a t i s t i c s . Let S n denote a s ta tistic having an asy m p to tic S ta n d a rd N orm al d istrib u tio n , for exam ple, S n = (9—0) / s, w here i is a consistent e stim a te of th e a sy m p to tic variance of 0. If we d enote th e j ’th cu m u lan t of S n

by Kj>n, th e n o ta tio n in d icatin g its dependence on n, an d if r/>n denotes th e c h a ra c te ristic fu n ctio n of 5 n , th e n we have

ipn(t) = E {exp ( i t S n )}

= exp{/€ifUit

+ ^K2,n(it)2

4---b

jiKjiTl( i t y

4---}. (1.16)

In m an y cases of p ra c tic a l in te re st, KjiTl is of o rd e r n ~ ^ ~ 2^ 2 a n d m ay be e x p a n d e d as a pow er series in n ~ l :

K j , n = n ~ {j~2)/2(kjfl + n ~ 1 k j i2 4--- ), j > 1, (1-17)

w here k \ ^ = 0 a n d &

2

,i = 1, these relatio n s reflecting th e fact th a t S n has asy m p to tica lly zero m e an a n d u n it variance; see Jam es (1955, 1958) an d Jam es a n d M ayne (1962). In th e sim pler case w here S n is a n o rm alized su m of in d e­ p e n d e n t an d id en tically d is trib u te d ra n d o m variables, fo rm u la (1.17) is im plicit in th e first id e n tity of (1.6), w here Kj^n = n - 0’- 2 ) /2kj, for j > 2, w here kj is th e j ’th c u m u la n t of Y . C om bining (1.16) a n d (1.17) we see th a t

^ n ( t ) = e x p [ - ^ f 2 + n ~ ^ { k i i2H

4-4- n 1 { \ k 2^2(it)2 4- jr& 4 ,i(^ )4} 4- • •• ] = e -“^ 2 ( l 4- n ~ * { k i a it 4- |Ar3>i ( i t ) 3}

+ n ~ l [kk2,2{it)2 4- ^ ^ 4 ,i(® 0 4 4" 4- |& 3ti(z£)3 }2]

4 - 0 ( n ' 3/ 2)). (1.18)

M ore generally,

= e_ 2^2( i

4

-

n~^ r \ ( i t )

4-

n ~ l r2{it)

4--- b

n ~ 3^2Vj(it)

4--- ),

w here

rj

is a p o ly n o m ial of degree no m ore th a n 3j , a n d is an even or o d d fu n ctio n according as j is even o r odd. T h is ex p an sio n is form ally id en tical to (1.8), w hich was derived in th e special case w here S n is a su m of in d e p en d e n t ra n d o m variables, a n d m ay b e in v erted to o b ta in an analogue of (1.14):

w here pj is a poly n o m ial of degree no m ore th a n 3j — 1, a n d is o d d /e v e n for e v e n /o d d j .

R ecall from Subsection 3.2 th a t

r o o

/ e' txd { - H e j - i { x ) ( f ) ( x ) } = (U)j e ~ ^ t , j > 1,

J — oo

w here H e j is th e j ’th H erm ite polynom ial. From th is fact, a n d in v ertin g (1.18), we get

Pi ( x ) = —{ ^ i,2 + ^ h A H e 2( x ) } = - { k 1>2 4- ^ 3 , i ( z 2 - 1)}, (1-20) an d

p 2(x) = - { f ( f c2,2 + k l 2) H e i ( x ) + ^ ( f c 4,i + 4 k i t2k3ti ) H e s ( x )

+ h kl , i H e ^ x )}

= —x { ^ ( k 2'2 + k \ 2) + ^ ( ^4,1 + ^ k i i2k 3' i ) ( x2 — 3)

+ i j f c ^ x 4 - I 0 x 2 + 15)}. (1.21)

W e in te rp re t ex p an sio n (1.19) as an a sy m p to tic series; th a t is,

P ( 5 n < x) = $(ic) 4- n~^pi(x)<f>(x) 4 - --- f n ~ J^2pj(x)(f)(x) 4- o(rc-jf/ 2),

for j > 1, uniform ly in x u n d e r a p p ro p ria te re g u la rity con d itio n s on X . We discuss su ch re g u la rity con d itio n s in m ore d e tail in th e n ex t su b sectio n .

N ext we illu s tra te th e a p p lic atio n of E d g ew o rth expansions to describing coverage accu racy of confidence intervals for a p a ra m e te r 9. Suppose th a t we define S n = (9 — 9 ) / cr , w here n ~ 1cr2 is th e a sy m p to tic variance of 9. T h e n one- a n d tw o-sided n o m in al (1 — a)-lev el N o rm al-th eo ry confidence in terv als for

0 are respectively

Ji = ( —00, 9 + n ~ 2 ( J Z i - a ) a n d I 2 = (9 - n ~ 1 crzl _ ( a / 2) , 9 4- n 2<72i_(a / 2)), w here z7 denotes th e so lu tio n of th e e q u atio n $ ( z 7 ) = 7, for 0 < 7 < 1. B earing in m in d th a t e v e n /o d d in d ex ed p f s are o d d /e v e n fu n ctio n s respectively, we see from ex p an sio n (1.19) th a t th e a c tu a l coverage p ro b ab ilities of th ese intervals are

P (0 G I \ ) = P (0 < 9 4- n ~ ^ ( TZ i - a ) = P ( 5 „ > - z i - a )

= 1 - { ^ ( -2T!_a ) 4- n ~ ^ p i ( - 2 : i _ a ) ^ ( - z i _ a ) 4" 0 ( n - 1 )}

an d

P (# £ h ) = P{ S n > — Z l - ( a / 2 ) ) ~ P(*Sn > z l - ( a / 2 ) )

= P { S n < z l - ( a / 2 ) ) — P ( S n < — z l - ( a / 2 ) )

= $ ( Z l - ( a / 2 ) ) ~ $ { — z l - ( a / 2 ) )

+ n 2 { P l ( z l - ( a / 2 ) ) (f) ( z l - ( a / 2 ) ) ~ P i z l - ( a / 2 ) )4>( ~ Z1 - ( a / 2 ) ) }

+ n ~ 1 { P 2 ( z l - ( a / 2 ) ) ( P ( z l - ( a / 2 ) ) ~ P 2 ( - z l - ( a / 2 ) ) < l > ( - z l - ( a / 2 ) ) }

+ ^ _ 3 / 2 { P 3 ( ^ l - ( c r / 2 ) ) ^ ( - l - ( a / 2 ) ) ~ P 3 ( ~ ^ 1 - ( a / 2 ) )<t>(~z l - ( a / 2 ) ) } + 0 ( n " 2)

= 1 - a + 2 n _1p 2( ^ i - ( a / 2 ) ) 0 ( ^ i - ( a / 2 ) ) + 0 ( n - 2 ). (1.23)

In p a rtic u la r, one-sided N o rm al-th eo ry confidence in terv als have coverage erro rs of o rd e r n ~ 2, w hereas tw o-sided in terv als have coverage erro rs of o rd er

n ~ l . U sing (1.19) a n d th e arg u m e n t w hich we em ployed to o b ta in (1.22) a n d (1.23), it follows th a t, m o re generally,

P(0 e h ) = I - a + n ~ j / 2 ( - l ) JP j ( z i-a)<i>(z i - a ) , (1.24) a n d

P (0 £ h ) = 1 - a + 2 n ~ JP2j(z i-{*/2))<i>(z i-{o'/2)), (1.25)

w here, as u su al, we in te rp re t these expansions as a sy m p to tic series. T herefore, as a re su lt of th e cru cial p ro p e rty th a t e v e n /o d d indexed p j ’s are o d d /e v e n fu n ctio n s, respectively, we see th a t th e coverage p ro b a b ility of one-sided intervals m ay be expressed as a pow er series in n ~ ^, w hile th e coverage p ro b a b ility of two- sided intervals m ay be w ritte n as a series in n ~ 1. T h is fact will prove critical in C h a p te rs 3 a n d 4 w hen we discuss b o o ts tra p ite ra tio n , for it m eans th a t each ite ra tio n to reduce coverage e rro r reduces th e o rd e r of e rro r by a fa c to r of n ~ ^

for one-sided in terv als, b u t by a facto r of n ~ 1 for tw o-sided intervals.

T h u s far, o u r discussion has cen tred on th e n o n -S tu d en tize d s ta tis tic n 2 (0 —

0)/(T. If er is unkn o w n , a n d if we e stim ate it by <r, say, th e n it m akes sense to w ork w ith th e S tu d e n tiz e d sta tistic S n = n ? ( 0 — 0 ) / a , in ste a d of — 0)/cr.

T h e d is trib u tio n fu n c tio n of th e new S n a d m its an ex p an sio n of th e sam e form as (1.19), b u t w ith different polynom ials w hich we d enote by qj. H ence,

w here, as for th e p / s , e v e n /o d d indexed <?/s are o d d /e v e n fu n ctio n s respectively. E x am p les of one- an d tw o-sided nom in al (1 — a:)-level N o rm al-th eo ry confidence in terv als for 9 b ased on th e a sy m p to tic N orm al d is trib u tio n of S n are

respectively. T h e ir a c tu a l coverages are given by (1 .2 2 )-(1 .2 5 ), except th a t p / s th e re a re now rep laced by q f s.

W e m ay ap p ly E d g ew o rth expansions to m an y o th e r fu n ctio n als of th e dis­ trib u tio n of 9. A n exam ple app licab le to bias re d u ctio n , am ong o th e r th in g s, is th e use of E d g ew o rth expansions to o b ta in expansions of m om ents. N ote th a t if

r > 1 is an in teg er a n d we define S n = n ^ ( 9 — 9)/cr, th e n

P ( 5 „ > x) + ( - l ) r P ( S „ < - x ) = P(|JV| > x) - 2 V n^p2j{x)4>{x),

* —

w here N denotes a S ta n d a rd N orm al N (0 ,1 ) ra n d o m variable. C onsequently,

w here th e infinite series are to be th o u g h t of as asy m p to tic expansions. W e m ay w rite (1.26) m ore concisely as

E{(e-Ö)r}

= ( n - i a ) rE(5D

Now, w h en r is even,

a n d w h en r is odd,

P(5„ >

x)

+ (—l ) rP(5„ <

- x )

=

- 2 n

i V

n

J p 2j-i(x)(j>(x),

*— J> 1

' (n 2cr)r {E(iV r ) - 2 r Ylj>i n J /o°° ^ 1

P2j(x)<l>(x) d x }i

t even,

E { ( 0 - 0 ) r } = <

—2 (n 2(j)r r n2 J2j>i n 3 fo°° xT 1 P 2 j - i ( x )<t>(x ) dx, r o d d ,

(1.26)

E {(0 - 0 )r } = n - ! (r+1)/ 2l(Cl + n ' l c2 + n ~ 2c3 + • • • ) ,

d is trib u tio n of S n. T h e fact th a t these expansions of m o m en ts are series in n _1 ex p lain s w hy b o o ts tra p ite ra tio n to correct for bias reduces th e o rd er of bias by a fa c to r of n ~ l ra th e r th a n n~ i .

A rig o ro u s p ro o f of expansions such as (1.26) requires a n o n u n ifo rm e stim ate of th e fo rm

sup (1 4- |ar|)z

■ o o < x < o o

P( S n < x) - < $ ( x ) + ^ 2 n k/2Pk(x )(p{x )

= 0 ( n - ^ ' +1)/2),

f c = i

for larg e /, of th e difference betw een P( S n < x) an d its E d g ew o rth ex p an sio n up to th e y th term . A lth o u g h such calculations m ay b e carried o u t, ex p an sio n s like (1.26) a re often available m ore generally th a n are th e E d g ew o rth ex p an sio n s from w hich th e y axe derived. For exam ple, if 9 a n d 9 d enote p o p u la tio n a n d sam ple m ean s respectively, a n d if th e r ’th o rd er p o p u la tio n m o m en ts are finite th e n (1.26) h o ld s w ith o u t fu r th e r assu m p tio n s on th e p o p u la tio n , such as C ra m e r’s co n d itio n . For th is reaso n , a n d to m ake o u r ex p o sitio n clearer, th ro u g h o u t th is th esis we shall w ork only form ally w ith E d g ew o rth expansions w ith o u t giving rigorous proofs.

3 .4 . A m o d e l fo r v a lid E d g e w o r t h e x p a n s io n s . In th is sub sectio n we p ro ­ vide a m o d el u n d e r w hich th e E d g ew o rth expansions discussed in th e previous su b sectio n are valid. Let

X, Xi , X2, *

• • d enote in d e p en d e n t a n d id en tically dis­ trib u te d ra n d o m d-vectors w ith m ean fi a n d p u t

X

= n ~ 1 X u < i< n Let A : R d —►R b e a sm o o th fu n c tio n satisfying A (/i) = 0. T y p ically we shall b e looking a t fu n ctio n s such as A (x ) = {g(x) — / r( f i ) or A (x ) = {^(x) — <7( ^ ) } / r ( x ) , w here 9

=

g(fx) is th e p a ra m e te r e stim a te d by 9 =

^f(X),

a n d w here is th e a sy m p to tic variance of 712 9 a n d t(X ) is an e stim a te of t(/z). T h is m odel is ap p licab le to m an y p ro b lem s of p ra c tica l in te re st in clu d in g those w here 9 is a m ean , o r a variance, o r a ra tio of m eans o r variances, o r a difference betw een m ean s o r variances, or a co rrelatio n coefficient, etc., in all of w hich 9 h as th e form #(/i) for a sm o o th fu n c tio n g a n d a m ean vector \i.

D en o te th e z’th co m p o n en t of a d-vector v by a n d p u t Z = n s(X — fi).

W e w rite

a*...

i, =

(

8

*

/ d x ^

•••3x<0)A (x)|x=)i .

Since A{fi) = 0 a n d Z = Op{ 1), we have, by T aylor expansion,

w here

S„r = Y aiz(,)

+ " ' H E E

+ • ■

•

1 = 1 11 = 112 = 1

+ n -(r -l)/2 (r!) - l

Y ' - ' Y

°>

i

...iTZ iil)

■

■

■

Z (ir).

(1.2S)

*1 = 1 lr = l

T h e following th eo rem shows th a t th e cu m u lan ts of

Snr

a d m it ex p an sio n (1.17).

T H E O R E M 1 .1 .

(Hall (1989), Theorem 2.1.) Let

Z = n ^ ( X —

p), where

X

denotes the mean o f n independent random vectors distributed as

X

with finite

j •

r ’th mom ents and mean vector p. Put

UnT = Y b' Z{,) + n - *

Y

Y bi^ z ( h ) z ( h ) + • • •

x = l *1 = 1 *2 = 1

+ n “ (r_1)/2

Y

" •

Y

■

■

■

z M ,

* 1 = 1 *r = 1

fo r ß x e d a rb itra ry constants b. Then the j ’th cum ulant o f Unr has the form

Kj,nr

=

n ~ ( j ~ 2)/2( k

j>i

+ n ~ l k j ,2

+

n ~ 2k j j

H---- ),

where the constants kj,i depend only on the b’s, the m om ents o f

X

up to the

j

•

r ’th and r and where the series contains only a ßnite num ber o f non-zero

terms.

A p ro o f of T h eo rem 1.1 is given in H all (1989).

P ro v id ed th e first

j

m o m en ts of th e Op( n - r / 2) te rm in (1.27) are all of o rd er n “ r / 2, w hich is g u a ra n te e d u n d e r sufficiently strin g e n t m o m en t co n d itio n s on X a n d m ild assu m p tio n s a b o u t

A

, th e

j

’th c u m u lan t

KjjTl

equals th e

j ’th.

cu m u la n t of

Snr

u p to o rd er n - r / 2. T h is re m a in d er m ay b e m ad e a rb itra rily sm all by choosing r sufficiently large. T herefore,

Kj^n

ad m its a sy m p to tic ex p an sio n (1.17).

T h e n ex t step is to co m p u te th e variance of

S n

so th a t, u p o n rescaling,

S n

h as u n it asy m p to tic variance. T h e la tte r con d itio n is im p licit in (1.19). P u t

s E { (X - #.)“ *> • • • (X - p ) < 0 } ,

j >

1.

T h e n

ßi =

0 for each

i,

E

( Z ^ Z ^ ^ ) = ßij, E ( Z ^ Z ^ ) — n ~ i ß i j k ,

a n d

T aking r = 2 in (1.27) so th a t

d d d

Sn =

^ a , Z (,)

+ n - H +

0 p( n - ‘),

1=1 z=l .7 = 1

we have

E (Sn) = + 0 ( n _1 ),

E (‘S,n) =

Y l i Y l j ai aJ^J

+ ö (n_1)^

an d

ec

5*) =

n ~ ^

I

T , T j Tfc

a ' a i a * v < j k

+ t T / i 2 d T /fc T ; a 'al ak‘ + Pi + P ilP ji)

+ 0 { n ~ 1).

C onsequently, th e first th re e cu m u lan ts of 5 n axe

« i,n = E ( 5 n ) = n 2 A i + 0 ( n *), « 2,n = v a r(5 „ ) = cr2 + 0 ( n _ 1 ), and

K3,n = E ( S J ) - 3 E (S * )E (S „ ) + 2( E S „ ) 3 = n ~ U 2 + O( n “ 1), w here

<7 — ^ ajCLjfiiji (1.29)

A i = 2 y ^ QxjUij', (1.30)

If th e a sy m p to tic v arian ce of S n , cr2, is n o t u nity, th e n we redefine S n = n2 A (X )/<r. It th e n follows from th e above calcu latio n s th a t for th is definition of S n , th e c o n sta n ts £1,2 a n d £3,1 a p p ea rin g in fo rm u la (1.20) for p i ( x ) are

k i t2 = Ai<J-1 and fc3)i = A2cr~3.

T H E O R E M 1 .2 . (B h a tta ch a rya and G hosh (1978), T h eo rem 2.1.) A ssu m e th a t th e fu n c tio n A has j 4 - 2 continuous derivatives in a n eig h b o u rh o o d o f p = E (X ), th a t A ( p ) = 0, th a t E (||X ||-7+2) < oo, and th a t th e characteristic fu n c tio n ip o f X satisfies

lim s u p |?/>(t)| < 1. (1.32)

l | t | | - o o

D eh n e a{l ...ir , p l l ...ir , cr, A \ , an d A 2 as above. S u p p o se cr > 0. T h en fo r j > 1,

P { n2 A (X )/c r < a:} = <$(z) + n ~ *pi(x)<j>(x) + • • • + n _ J/2pj(x)<p(x) + o (n ~J/ 2),

(1.33)

u n ifo rm ly in x, w here p j is a p o lyn o m ia l o f degree at m o st 3j — 1, o d d fo r even j an d even fo r o d d j . In particular,

P i(x ) = - { A i d " 1 + l A 2cr~3( x 2 - 1)}. (1-34)

B h a tta c h a ry a a n d G hosh give a p ro o f of th is resu lt. C o n d itio n (1.32) is a m u ltiv a ria te form of C ra m e r’s condition, discussed earlier. H all (1989) proves th a t it is satisfied if th e d istrib u tio n of X h as a n o n d eg en erate a b so lu tely con­ tin u o u s com ponent.

It rem ain s to discuss m odels in w hich cr2 is unknow n, a n d we m u st e stim a te it by <t2 = t2(X ). We m u st first verify th a t th e a sy m p to tic variance of an e stim a to r

0 = </(X) of 9 = g( p) m ig h t b e tak en equal to a fu n ctio n r 2 of th e m e an p an d m ig h t be e stim a te d by th e sam e fu n ctio n of X . If A (x ) = g (x ) — g(p) , th e n by (1.29),

1=1 j=l

w here g(i)(x) = ( d / d x ^ ) g ( x ) . Suppose we ad jo in to th e vector X all those p ro d u c ts for w hich g ^ ( p ) g ^ p ( p ) 0, an d w hich do n o t alread y a p p e a r in X , an d ad jo in analogous te rm s to th e vectors X* a n d X . Let p d e n o te th e m e an vector of th e new , len g th en ed X , a n d p u t X = n -1 • X* . T h e n cr2 is a fu n ctio n of p, say r 2(/i) a n d is e stim a te d A /n-consistently by t2(X ). We have, by analogy w ith T h eo rem 1.2, th e following a sy m p to tic ex p an sio n for th e d is trib u tio n fu n ctio n of S n = n ^ A ( X ) /d :

P { n2 A (X )/<r < x ] = <£(x) -f n * qi(x)</>(x)-\--- + n qj(x)<j>(x) 4- o(n J^ 2),

u n ifo rm ly in x. w here q3 is a polynom ial of degree a t m ost 3j — 1, od d for even j

a n d even for o d d j . If T h eo rem 1.2 is ap p lied to th e S tu d en tized m ean, expansion (1.35) holds assu m in g (2j 4- 2 )’th m om ents on th e u n d erly in g d istrib u tio n . Hall (1987c) gives a m ore refined arg u m en t m aking extensive use of th e special p ro p e r­ ties of th e S tu d e n tiz e d m ean to show th a t (j + 2 )’th m om ents suffice in th a t case. M ore generally, if we define B = A / r , = B l l ...ir(p), # i = | bijPij

a n d

B i = bibjbklJijk + 3 T t E ,

we have qq (a;) = — { B \ + |i ? 2 ( £ 2 — 1)}; see H all (1988a), Section 1.4. H all (1988a) also shows th a t p i ( x ) — qi ( x) = — |< r- 3 ( J T S j aiCjPijX2), w here C{ = g ^ ( p ) .

T h u s far in th is section we have m ad e little m en tio n of th e b o o ts tra p . How­ ever, th e p resen t m odel for S n fits well in to th e general fram ew ork for b o o ts tra p resam p lin g developed in C h a p te r 3. T h e reaso n for th is is th a t /i, a n d hence equals a fu n c tio n a l #(•) of th e p o p u la tio n d is trib u tio n fu n ctio n F. In all n o n p a ra m e tric cases, a n d in m an y p a ra m e tric exam ples, g (X ) is th e sam e func­ tio n a l of th e em piric d is trib u tio n fu n ctio n F . T h is fact will prove v ita l to our developm ent of E d g ew o rth expansions in th e b o o ts tra p co n tex t.

3 .5 . C o r n is h -F is h e r e x p a n s io n s . Let S n d en o te a sta tistic having a d istrib u ­ tio n a d m ittin g E d g ew o rth expansion (1.19), w here we in te rp re t th is expansion as an a sy m p to tic series. W rite ic i_ a = w i - a (n) for th e (1 — a)-lev el q u an tile of

Sn,

P(Sn < W\ —a) = 1 - a,

a n d let z \ - a b e th e (1 — o)-level q u an tile of th e S ta n d a rd N orm al d istrib u tio n . We m ay in v ert ex p an sio n (1.19) to o b ta in an ex p an sio n of w \ - a in te rm s of z \ - a, a n d vice-versa:

w i - a

=

z

i_a +

n ~ ^ p i i ( z 1- a )

+

n ~ 1P 2 i ( z i - a )

H

----+ n ~ j / 2p j i ( z i - Q)-\--- , (1.36) a n d

Z i - a = W i - a + n "^ p12(z 1_ a ) + n ~ l p 2 2 ( z i - a )

H---+ n ~ j / 2p j 2( z 1- a ) H--- . (1-37)

0 < e < T h e polynom ials pj \ an d p j2 are of degree a t m ost j + 1 and are o d d /e v e n fu n ctio n s for e v e n /o d d j . T h ey are d eterm in ed com pletely by th e pj' s

in (1.19) a n d satisfy th e form al relatio n s

1 - a = $ j - z i - a + X ^ > i n ~ ] /2P j i ( z i - o )

+ E , > 1 " " ' /2P‘ { Zl - « + £ ; >

x < j ) ^ z i - a + y ; . >i n - j / 2 p j i (2Ti_a ) | , (1.38)

for 0 < a < 1, an d

E i>x n _ ,/2 P j i ( x ) + E , > 1 n _ ,/2 P<2 n “ -,/2p j i ( x ) | = 0. (1.39)

In p a rtic u la r, T ay lo r-ex p an d in g th e le ft-h a n d side of (1.38) a b o u t z \_ a , we get

1

- a =

1

— a +

n ~ ^ { p u ( z 1- a )

+

P i(2 i-a )} < K zi - a )

+ n ~ l [ p 2 l ( * l - a ) ~ 5 P u ( z i - a ) z i - a

+ P l l ( z i - a ) { p ' i ( z i - a ) - P i ( z i - a ) z i - a }

+

P2( z i - a ) \ fit*

i - a ) +

0 ( n ~ 3/2),

fro m w hich we conclude th a t

P n ( x ) = - p i { x ) a n d p 2i( x ) = p i ( x ) p [ ( x ) - \ x p i ( x )2-

p2(^)-If th e d is trib u tio n of S n a d m its E d g ew o rth ex p an sio n (1.35) r a th e r th a n (1.19) (for exam ple, if a is u nknow n a n d we e stim ate it by or), th e n expansions (1.36) a n d (1.37) hold, except th a t polynom ials pj i an d p j 2 th e re are replaced by different p olynom ials, w hich we d en o te by qj\ a n d qj2 respectively. R elations (1.38) an d (1.39) th e n h o ld w ith q’s in ste a d of p ’s a n d we deduce th a t

<hi(x)

=

- q i ( x ) a n d q2\ { x) = qi ( x ) q[ ( x ) - \ x q 1( x )2 - q2(x).

In m ost of o u r w ork we req u ire only th e first tw o p olynom ials in C ornish- F ish e r inversions of (1.19) a n d (1.35). T h e polynom ial qzi , w hich we need in C h a p te r 2, is derived in A p p en d ix A2.3.

T h e id e a of e x p an d in g a d istrib u tio n function in w h at we now call an Edge- w o rth ex p an sio n o rig in ate d w ith th e work of C hebyshev (1890) a n d E d g ew o rth (1904), w ho discussed form al c o n stru ctio n of expansions for sum s of in d e p en d e n t ra n d o m variables. C ra m er (1928; 1946, C h a p te r V II) gave rigorous th e o ry in th a t case. L a te r work in th is a re a was carried o u t by G eary (1936, 1947), G ayen (1949, 1950) a n d W allace (1958), before form al expansions were p u t on a rigorous fo o t­ ing, larg ely by th e w ork of H su (1945), Bickel (1974), C hibishov (1972, 1973a,

1973b), a n d B h a tta c h a ry a an d G hosh (1978), am ong o th ers. See also C h am b ers (1967), P fan zag l (1973a, 1973b, 1979), D avis (1976), S arg an (1976) a n d P h illip s (1977).

O u r developm ent of E d g ew o rth ex p an sio n th eo ry has relied heav ily on th e basic cu m u la n t ex p an sio n (1.17). F ish er (1930) recognized th a t fo rm u la w ith o u t giving a com prehensive proof, a n d Jam es (1955, 1958) an d Ja m e s a n d M ayne (1962) h ave given m o re th o ro u g h a rg u m en ts. See also K en d all an d S tu a r t (1977), C h a p te rs 12 a n d 13.

P e tro v (1975), C h a p te r V I has given a d etailed account of E d g e w o rth e x p an ­ sions for d is trib u tio n s of sum s of in d e p en d e n t ra n d o m variables, a n d his Section 1, C h a p te r VI is a useful in tro d u c tio n to th e polynom ials a p p e a rin g in these ex p an sio n s.

C ra m e r’s co n d itio n h as played a m a jo r role in th e th e o ry of E d g ew o rth ex p an sio n s, even in th e sim p lest cases. E sseen (1945) show ed t h a t o n e-term ex p an sio n s w ith re m a in d e r o (n - ^) are valid u n d e r th e w eaker co n d itio n of non- la ttic en e ss, an d he also tre a te d expansions for la ttic e ra n d o m variables. B h a t­ ta c h a ry a a n d R ao (1976), C h a p te r 5 have pro v id ed a com prehensive tre a tm e n t of E d g e w o rth ex p an sio n s for sum s of in d ep en d en t lattice-v alu ed ra n d o m variables.

B h a tta c h a ry a a n d R ao (1976) also gave an excellent tre a tm e n t o f expansions in th e case of sum s of vector-valued ra n d o m variables. T h a t w ork fo rm ed th e b asis for B h a tta c h a ry a a n d G h o sh ’s (1978) rigorous tre a tm e n t of E d g ew o rth exp an sio n s. W ith e rs (1983, 1984) p re sen te d general form ulae for early te rm s in E d g e w o rth ex p an sio n s u n d e r m odels inclu d in g th a t tre a te d in S u b sectio n 3.4. Inverse (C o rn ish -F ish e r) expansions were first stu d ie d by C o rn ish a n d F ish er (1937), a n d have b een discussed by K en d all a n d S tu a rt (1977), Section 16.21, a n d H all (1983).