Semiparametric Likelihood Ratio Inference

SEMIPARAMETRIC LIKELIHOOD RATIO INFERENCE

By S. A. Murphy1_{and A. W. van der Vaart}

Pennsylvania State University and Free University Amsterdam

Likelihood ratio tests and related confidence intervals for a real pa-rameter in the presence of an infinite dimensional nuisance papa-rameter are considered. In all cases, the estimator of the real parameter has an asymptotic normal distribution. However, the estimator of the nuisance pa-rameter may not be asymptotically Gaussian or may converge to the true parameter value at a slower rate than the square root of the sample size. Nevertheless the likelihood ratio statistic is shown to possess an asymp-totic chi-squared distribution. The examples considered are tests concern-ing survival probabilities based on doubly censored data, a test for presence of heterogeneity in the gamma frailty model, a test for significance of the regression coefficient in Cox’s regression model for current status data and a test for a ratio of hazards rates in an exponential mixture model. In both of the last examples the rate of convergence of the estimator of the nuisance parameter is less than the square root of the sample size.

1. Introduction. In the past decade considerable progress has been made with the study of maximum likelihood estimators in infinite dimensional sta-tistical models, sometimes called nonparametric maximum likelihood estima-tors (NPMLE) or semiparametric maximum likelihood estimaestima-tors. See, for instance, Gill (1989) or van der Vaart (1994a) for reviews of work in this direc-tion and Gu and Zhang (1993), Huang (1996), Murphy (1995a), Van der Laan (1993), van der Vaart (1994c, 1996), Gill, van der Laan and Wijers (1995), Huang and Wellner (1995) and Wijers (1995) for more recent results. Most of this work is directed at proving the asymptotic normality and efficiency of the maximum likelihood estimator of smooth parameters of the model. In contrast, very little progress has been made toward a general likelihood ratio theory for semiparametric models. Here the term ‘semiparametric model’ is used in a loose sense as a model which is not finite dimensional (as in classi-cal statistics), nor fully nonparametric [cf. Bickel, Klaassen, Ritov and Wellner (1993), pages 1–2]. In this paper we give a general approach for the asymp-totic analysis of hypothesis tests and associated confidence regions based on the (semiparametric) likelihood ratio test.

The results of this paper can be viewed as a step in filling the large gap be-tween classical parametric likelihood ratio theory and empirical likelihood as considered by Thomas and Grunkemeier (1975), Owen (1988), Qin and Law-less (1994) and Murphy (1995b) among others. These authors are concerned with the situation where the model for the data is fully nonparametric, in the sense that it contains every possible probability distribution on the sample

Received September 1995; revised September 1996.

1_{Research partially supported by NSF Grant DMS-93-07255.}

AMS1991subject classifications. 62G15, 62G20, 62F25.

space, or is restrained by finitely many constraints [as in Qin and Lawless (1994) ifr > p]. Each time the “likelihood” is taken as the productQ_iP_”X_i•

of the masses given to the observational points, referred to as the empirical likelihood. Qin (1993) and Qin and Wong (1996) extend the above theory to specific semiparametric models. However, in all of the above cases, the likeli-hood ratio statistic reduces to a function of a vector statistic (often a Lagrange multiplier), simplifying the asymptotic analysis greatly. General semiparamet-ric models, of the type we consider in this paper, appear to require a different approach. For example, this simplification will not occur in the mixture model considered by Roeder, Carroll and Lindsay (1996) in which they invert a like-lihood ratio test to form a confidence interval for a regression coefficient.

In the classical parametric case likelihood ratio confidence regions are gen-erally preferred over Wald-type confidence regions, except perhaps from a computational perspective. Advantages mentioned by many authors are small sample coverage probabilities closer to the nominal values, the possibility of asymmetric confidence regions and regions that are transformation respect-ing [Hall and La Scala (1990)]. Although we do not give a proof in this paper, many of these advantages may be expected to carry over to the semiparamet-ric situation. This appears to be particularly the case when the small sample distribution of the estimator is highly skewed. A concrete example in the non-parametric setting is in the construction of confidence intervals for a survival probability based on the Kaplan–Meier estimator. The Wald confidence inter-val for a surviinter-val probability performs poorly and much work has been done on finding a transformation of the estimator which has an approximate normal distribution for small samples [Andersen et al. (1993)]. In this case inversion of the likelihood ratio test as in Thomas and Grunkemeier (1975), Li (1995) and Murphy (1995b) illustrate how the resulting confidence intervals perform as well as confidence intervals based on widely accepted “best” transforma-tions of the estimator. In their comparison of empirical likelihood confidence intervals with bootstrap confidence intervals, Hall and La Scala argue, that in situations in which both methods can be applied, the empirical likelihood confidence intervals are to be preferred to bootstrap confidence intervals. They state that “the power of the bootstrap resides in the fact that it can be applied to very complex problems and this feature is not available for empirical like-lihoods.” Usually inference for semiparametric models is a complex problem; however, as we shall demonstrate, likelihood ratio inference will, in general, be available.

involve inverting a matrix of high dimension. This is true, for instance, in the semiparametric frailty model considered by Nielsen, Gill, Andersen and Sorensen (1992) and Murphy (1995a), where estimators for the standard error of the estimated frailty variance are found by inverting a matrix which is of the same dimension as the data.

Furthermore, in cases where the efficient influence function can be written down relatively explicitly, the estimation of its variance may involve nonpara-metric smoothing. This means that the researcher must deal with the difficult choice of a smoothing parameter. This is the case in estimating the regression coefficient in Cox’s regression model subject to current status type censoring, considered by Huang (1996), where the efficient influence function depends on a ratio of conditional means.

For the above reasons setting Wald-type tests and associated confidence regions in semiparametric models may be computationally harder than in the classical situation, where one can use a plug-in estimator based on an expression for the Fisher information or the observed information. Therefore, due to the expected gain in quality, likelihood ratio based inference appears doubly attractive in semiparametric settings.

The definition of a likelihood ratio statistic requires the definition of a like-lihood function. In classical parametric models this is the density of the ob-servations, while empirical likelihood theory uses the product QP_”X_i•. We do not offer a general definition of an infinite dimensional likelihood function in this paper. In some examples the observations have a well-defined density and a likelihood is defined much as in the classical situation. In other ex-amples one uses the empirical likelihood (which, however, is maximized only over the model). Mixtures of these situations occur as well in the literature and in some missing data situations a “partial likelihood” appears appropri-ate. Some of these possibilities are illustrated in our four examples. With this formulation a semiparametric likelihood estimator is not necessarily discrete (although it can often be taken to be discrete) and it is often not supported on the observed data. Restricting the estimator to a null hypothesis may intro-duce new support points.

We consider the situation that the observationsX₁; : : : ; X_n are a random sample from a distribution P_ψ indexed by a parameter ψ that is known to belong to a set 9. Given a parameter (map) θ_x 9 _{→ R} and a definition of a likelihood lik_ψ; X_‘ for one observation, the likelihood ratio statistic for testing the null hypothesisθ_ψ_{‘ =}θ₀ is given by

lrt_nθ_{0‘ =}2

sup

ψ∈9 n X

i=1

ln lik_ψ; X_{i‘ −} sup

ψ∈9; θψ‘=θ0 n X

i=1

ln lik_ψ; X_i‘

=2n_Pnln lik_{ ˆ}ψ_{‘ −}2n_Pnln lik_{ ˆ}ψ_0‘:

Here_Pn is the empirical distribution of the data andψˆ andψˆ0 are the

maxi-mum likelihood estimators under the full model and the null hypothesis, re-spectively. In the first exampleψis a distribution functionFandθψ‘ =Ft0‘

ψtakes the formψ= θ; 3‘orψ= θ; F‘for an unknown cumulative hazard function3or distribution functionFandθ_ψ_{‘ =}θ.

For simplicity we restrict ourselves to one-dimensional parametersθ. Then we wish to prove that under the null hypothesis the sequence lrt_nθ_0‘ con-verges in distribution to a χ2_{-distribution on one degree of freedom. Having}

proved this for every value ofθ₀, the region_”θ_xlrt_nθ_{‘ ≤}z2_α/2•is the associated confidence region of asymptotic level 1₋α.

The organization of the paper is as follows. In Section 2 we present four rather different examples for which we discuss the meaning of the likelihood and state theorems on the likelihood ratio test. The examples include the double censoring model considered in Chang (1990), regression for current status data considered by Huang (1996), the gamma frailty model of Murphy (1994, 1995a) and a mixture model studied by van der Vaart (1996). Section 3 starts with a discussion of the finite dimensional situation to gain intuition and next gives a general approach to prove the asymptotic validity of the semiparametric likelihood ratio test. The basic scheme given by Theorem 3.1 leaves some nontrivial work for special examples. However, our impression is that it works in the situations where also the asymptotic normality of the maximum likelihood estimator of the parameter of interest can be proved. The last sections contain detailed treatments of our four examples.

2. Examples and results. This section contains four examples. For each example we give the definition of the likelihood and state a theorem on the likelihood ratio statistics. Proofs are given in Sections 4–7.

Example(Doubly censored data). Doubly censored data arise when event times are subject to both right and left censoring. The event timeTis observed only if it falls between the left and right censoring times,LandR. Otherwise all that is observed is L and that T _≤ L in the case of a left censoring or

R and that T > R in the case of a right censoring. It is assumed that T

is independent of _L; R_‘ and that L_≤ R. Thus the observations are n i.i.d. copies of X_{= }U; D_‘, whereU₌Land D₌1 ifT_≤L,U₌TandD₌2 if

L < T_≤RandU₌RandD₌3 if T > R. If G_L andG_R are the marginal distributions of L _≤ R and F the distribution of T, then, with lowercase symbols denoting densities, the density ofXis given by

p_FX_{‘ = ’}F_U_‘g_LU_‘“I”D=1•_’f_U_‘G_L−G_R‘U_−‘“I”D=2•

× ’1₋F_U_‘‘g_RU_‘“I”D=3•:

WhenFis completely unknown, the above density is not suitable for use as a likelihood. Instead we use the empirical likelihoodP_F”X_•, which is obtained by replacing the densities g_L, f and g_R by the point probabilities G_L”U_•,

F_”U_•andG_R”U_•. For inference aboutFwe can drop the terms involvingG_L

andG_Rand define the likelihood to be

We maximize the above “likelihood” over discrete distribution functions with steps at the U’s. The parameter of interest will be θ_F_{‘ =}Fg ₌ Rg dF for some known functiong of bounded variation. Of particular interest isg_t_{‘ =}

1_”t > t_0•, which leads to a confidence set for 1₋F_t_0‘, the probability of survival longer than t₀. The concavity in F of the density in X along with the continuity of θ in F implies that the confidence set will be a confidence interval (see the Appendix).

We shall prove the following theorem. (Note that denotes convergence in distribution throughout this paper.)

Theorem 2.1. Suppose that_G_L−G_R‘u_{−‘ =}P_L < u_≤R_‘is positive on the convex hull_’σ; τ_{“ ⊂ ’}0;_∞‘of the support ofF₀. Furthermore, assume that

F₀, G_L andG_R are continuous, with G_Lτ_{‘ =}1 andG_Rσ_{−‘ =}0. Let gbe a left continuous function of bounded variation which, on_’σ; τ_“, is notF₀-almost everywhere equal to a constant. IfF₀g₌θ₀, then the likelihood ratio statistic for testing thatFg₌θ₀ satisfieslrt_nθ_0‘ χ2₁underF₀.

The asymptotic consistency and normality of the unrestricted maximum likelihood estimator in this model was proved under stronger conditions by Chang and Yang (1987), Chang (1990) and under weaker conditions by Gu and Zhang (1993).

Example(Cox regression for current status data). In current status data,

nsubjects are examined each at a random observation time and at this time it is observed whether the event time has occurred or not. The event timeTis assumed to be independent of the observation timeYgiven the covariateZ. Then the observations areni.i.d. copies ofX_{= }Y; δ; Z_‘, whereδ₌1 ifT_≤Y

and zero otherwise. Suppose that the hazard function ofTgivenZ₌zis given by Cox’s regression model: the hazard at timetiseθz_{λt‘. Then the cumulative}

hazard at timet of Tgiven Z=z is of the form eθzRt

0λs‘ds=eθz3t‘ and

the density is given by

p_{θ; 3}X_{‘ =} 1₋exp_−exp_θZ_‘3_Y_‘‘δ exp_−exp_θZ_‘3_Y_‘‘1−δfY; Z_Y; Z_‘:

The parameter of interest is the regression parameterθ; the nuisance param-eter 3 is assumed completely unknown. A test of regression would be a test of H0xθ=0.

The likelihood lik_θ; 3; X_‘is taken equal to the density, but with the term

fY; Z_{Y; Z‘omitted. To estimateθ and3 we maximize the likelihood overθ}

in a bounded parameter set2_⊂Rand over3ranging over all nondecreasing cadlag functions taking values in _’0; M_“, for a knownM.

We shall prove the following theorem.

Theorem 2.2. Letθ0be an interior point of2. LetYhave a Lebesgue

and 30τ‘ < M, and zero otherwise. Let30 be differentiable on this interval

with derivative bounded away from zero. LetZbe bounded andEvar_Z_ŽY_‘>

0. Finally assume that the function h∗∗ _{given by (5.1) has a version which is}

differentiable with a bounded derivative on _’σ; τ_“. Then the likelihood ratio

statistic for testingH_0xθ₌θ₀satisfies lrt_nθ_0‘ χ2

1 underθ0; 30‘.

The asymptotic normality of the maximum likelihood estimator for θ in this model is considered by Huang (1996). The maximum likelihood estimator for the cumulative hazard function converges at an O_n−1/3_{‘ rate in an L}

2

-norm. Under the hypothesis H_0x θ ₌ 0 this model reduces to the “case 1 interval censoring” considered by Groeneboom (1987), who obtains the limit distribution of Fˆ0t‘ for the distribution function corresponding to 3ˆ0 (with

an n−1/3_{-standardization). See Groeneboom and Wellner (1992).}

Example (Gamma frailty). In the frailty model, subjects occur in groups such as twins or litters. To allow for a positive intragroup correlation in the subjects’s event times, subjects in the same group are assumed to share the same frailtyZ. In the one-sample problem, we observe ni.i.d. groups where for a given group the observations are Jand_Tj∧Cj; Dj‘for j=1; : : : ; J,

where T_j is the event time associated with thejth subject in the group,C_j

is censoring time, D_{j =} 1 if T_{j ≤} C_j and Jis the random group size. The unobserved frailty Z is assumed independent of J and to follow a gamma distribution with mean 1 and variance θ. Given J, _C_j; j ₌ 1; : : : ; J_‘ is assumed independent of both Zand_T_j; j₌1; : : : ; J_‘. Given ZandJ, the

T_j; j₌1; : : : ; J_‘are independent, with hazardsZλ_·‘; j₌1; : : : ; J. PutN_t_{‘ =}P_jI_”T_j∧C_j≤t; D_j=1_•andY_t_{‘ =}P_jI_”T_j∧C_j≥t_•. So the observation for a group isX_{= }N; Y_‘. For our statistical inference we shall only use the values of this counting process on a given finite interval _’0; τ_“. Since the censoring is independent and noninformative of theZ, we have that given Z ₌ z, the intensity of N at time t is zY_t_‘λ_t_‘ and the conditional density is proportional to

p3XŽZ=z‘ = Y

t≤τ

zYt‘λt‘‘1Nt‘_exp

−zZτ 0 Y d3

;

where 3_{·‘ =} R₀·λ_s_‘ds. [See Andersen et al. (1993), pages 138–150, and Nielsen, Gill, Andersen and Sorensen (1992).] To form the marginal density for a group multiply by the gamma density ofZand integrate overzto get

p_{θ; 3}X_{‘ =} Q

t≤τ1+θNt−‘‘Yt‘λt‘‘1Nt‘

1₊θR₀τYt‘d3t‘‘1/θ+Nτ‘ :

When3is unknown, the associated likelihood has no maximizer. A convenient extension is

lik_θ; 3; X_{‘ =} Q

t≤τ 1+θNt−‘‘Yt‘13t‘ 1Nt‘

1₊θR₀τY_t_‘d3_t_‘1/θ+Nτ‘ :

This is not the only possible extension [see Andersen et al. (1993) and Murphy (1995a)]. We are particularly interested in an hypothesis test of zero intra-group correlation, that is,H_0xθ₌0.

The likelihood is also well-defined for negativeθclose to zero, even though

θ can then not be introduced through a gamma variable as previously. We define the likelihood ratio statistic and the maximum likelihood estimators relative to the parameter set consisting ofθranging over the interval_’−ε; M_“

for a small ε >0; and 3 ranging over all finite nondecreasing functions on

’0; τ_“.

Theorem 2.3. Assume thatθ_{0∈ ’}0; M_‘ and that3₀is continuous, strictly increasing and finite on_’0; τ_“. Furthermore, assume thatJhas finite support,

P_0’SJ_j=1”C_{j ≥}τ_•“>0, and that the distribution of _C_j; j₌1; : : : ; J_‘has at most a finite number of discontinuities. Then the likelihood ratio statistic for testingH_0xθ₌θ₀satisfies lrt_nθ_0‘ χ2₁ under_θ₀; 3_0‘.

The maximum likelihood estimator _{ ˆ}θ;3ˆ_‘ for this model was shown to be asymptotically consistent and normal by Murphy (1994, 1995a) under slightly more general conditions.

Example (Mixture model). This is another version of the frailty model. The group size is 2. As before, we allow for intragroup correlation in the event times by assuming that the pair share the same unobserved frailty

Z. GivenZ, the two event times T1 andT2 are assumed to be independent

and exponentially distributed with hazard rates Z and θZ, respectively. In contrast to the gamma frailty model, the distribution Fof Zis a completely unknown distribution on _0;_∞‘. The observations are n i.i.d. copies of X₌

T₁; T_2‘from the density

p_{θ; F}X‘ =Z zexp_−zT1‘θzexp−θzT2‘dFz‘:

We use this as the likelihood lik_θ; F; X_‘ and are interested in a confidence set for the ratio θof the hazards and testing that this ratio equals 1.

Theorem 2.4. Suppose thatR_z2_+z−6:5_‘dF

0z‘<∞. Then the likelihood

ratio statistic for testingH_0xθ₌θ₀satisfies lrt_nθ_0‘ χ2

1 underθ0; F0‘.

The maximum likelihood estimator for θ was shown to be asymptotically normal by van der Vaart (1996). The maximum likelihood estimator for the distribution function Fis known to be consistent from Kiefer and Wolfowitz (1956). Van der Vaart (1991) proved that the information for estimatingFt‘

is zero, so that the rate of convergence of the best estimators is less than the square root of n. A reasonable conjecture is that the optimal rate is n−α _for

3. Intuition. In the case that the parameterψis Euclidean a classical ap-proach to derive the asymptoticχ2_{-distribution of the likelihood ratio statistic}

is to expand the difference

2n_Pnlnp_ψˆ−lnp_ψˆ 0

in a two-term Taylor expansion around ψˆ. The linear term vanishes and al-gebraic manipulations involving the joint normal limit distribution ofψˆ−ψ0

andψˆ0−ψ0yield the result. A more insightful derivation can be based on the

approximation

ˆ

ψ_{− ˆ}ψ_{0= i}−1

ψ0θ˙0

˙

θT 0i−ψ01θ˙0

+ε

 ˆθ₋θ_0‘;

(3.1)

whereθ˙T₀i−_ψ1

0θ˙0is the asymptotic variance of

√_n

 ˆθ₋θ0‘;iψis the information

matrix,θ˙0is the derivative ofθψ‘with respect toψ; andεconverges to zero

in probability. If ψˆ is multivariate normal and θ is linear, then ε =0. More concretely, ifψ= θ; η‘andθψ‘ =θ, then

 ˆθ;η_ˆ‘T_{− }θ₀;η_ˆ0‘T₌ 1;₋i−_η0η01 i_η 0θ0+ε

T

 ˆθ₋θ_0‘;

(3.2)

where the information matrixi_ψ is partitioned into

i_ψ 0 =

_i

θ0θ0 iθ0η0 iη0θ0 iη0η0

:

Under regularity conditions both (3.1) and (3.2) can be justified by Taylor series arguments or by analogy to the case of a multivariate normal observa-tion [cf. Cox and Hinkley (1974), pages 308, 323]. If, as in Cox and Hinkley (1974), we neglect the error term ε, then we can replaceψˆ₀ in the likelihood ratio statistic by a constant timesθˆ₋θ₀, and next perform a two-term Taylor expansion in the one-dimensional parameterθˆ₋θ₀. This yields the approxi-mation

2n_Pnhlnp_θˆηˆ −lnp_θ0;ηˆ+i−1

η0η0iη0θ0‘T ˆθ−θ0‘ i

≈ −n ˆθ−θ0‘2Pn`¨·y ˆθ;ηˆ‘;

where `¨_·yt;η_ˆ‘is the second derivative of the mapt_→lnp_t;ηˆ+i−1

η0η0iη0θ0‘T ˆθ−t‘.

The first derivative of this function atθˆcan be expressed in the score functions,

 ˙`<sub>θ</sub>;`˙_η‘, for_θ; η_‘as

˙

`<sub>·y ˆ</sub>θ;ψˆ<sub>‘ = ˙</sub>`_{θˆ− }i−_η1 0η0iη0θ0‘

T_`˙ ˆ η:

By the usual identities the expectation of the second derivative should be mi-nus the expectation of the square of the first derivative and, under regularity conditions,

−Pn`¨<sub>·y ˆ</sub>θ;ψˆ<sub>‘ →P</sub>P<sub>θ0η0</sub> `˙_{θ0− }i−_η1 0η0iη0θ0‘

T_`˙ η0

2

This is exactly the _1;1_‘-element of the inverse of iψ0, which is the inverse

of the asymptotic variance of √n_{ ˆ}θ₋θ0‘. The chi-squared limit distribution

follows.

We might expect that, at least to the first order, the difference between the full and null maximum likelihood estimator in our semiparametric setting satisfies a generalization of (3.2). Then the difference is finite dimensional (the dimension of θ), and a standard Taylor expansion in θ can be used to prove the asymptotic chi-squared distribution. For example, Murphy [(1995b), equation (6) and the appendix] proves a result of this type for a likelihood ratio test based on the binomial likelihood for right-censored data. There,θ is the probability of survival past timet₀ and ψ is the cumulative hazard function

3, so thatθ₌θ_3_{‘ =}Q_s≤t

01−d3s‘‘. Fort≤t0,

ˆ

3_t_{‘ − ˆ}3_0t_{‘ =} Rt

0θ0/Ys‘ +λθ0‘d3ˆs‘

var_√n_{ ˆ}θ₋θ0‘‘ +ε

 ˆθ₋θ_0‘;

where Y_s_‘ is the number of individuals who have not failed or been cen-sored up to times, divided by the sample size and bothλ andε converge in probability to zero.

Whenψcan be estimated at a square rootnrate, then an analog toi−_ψ1exists and (3.2) can be extended appropriately. This approach can be used in both the gamma frailty and the double censoring examples. On the other hand, in many semiparametric models, including our current status and mixture examples, the nuisance parameter is not estimable at square rootnrate. Then an extension of (3.2) requires more care. Note for instance that (3.2) implies that the differenceη_{ˆ− ˆ}η₀ is of order O_n−1/2_{‘, often much smaller than the}

differencesη_ˆ−η₀andη_ˆ0−η₀(depending on the distance). In any case the two-step proof focusing first on the difference η_{ˆ− ˆ}η₀ and next on expanding the log likelihood requires careful choice of a norm in which the errorεis shown to converge to zero, since semiparametric likelihoods often contain ill-behaved terms.

In view of these potential difficulties our approach to proving the chi-squared limit distribution of semiparametric likelihood ratio statistics will be motivated by the approximation (3.2), but not based on it. Fundamental is the observation that the submodel t→p_t;η0−i−1

η0η0iη0θ0‘Tt−θ0‘ isleast favorable

at _θ₀; η_0‘ when estimating θ in the presence of the nuisance parameter η, in the sense that of all submodels t _→ p_t;η

t this submodel has the smallest

information aboutt. This information is precisely the asymptotic variance of

√_n

 ˆθ₋θ0‘. Thus 1;−i−η01η0iη0θ0‘ is the least favorable direction of approach

to _θ0; η0‘when estimatingθin the presence of the unknownη.

Approxima-tion (3.2) shows thatη_ˆ approachesη_ˆ0approximately along the least favorable

direction.

The derivative of the logarithm of the least favorable submodel with respect totat zero is called theefficient score functionand takes the form

˜

`<sub>θ0 = ˙</sub>`_{θ0− }i−_η1 0η0iη0θ0‘

The sequenceθˆis asymptotically linear in this function in that

√

n_{ ˆ}θ₋θ_{0‘ =}√n_Pn`˜_θ

0

/ i_θ

0θ0−iθ0η0i −1 η0η0iη0θ0

+o_P1_‘:

The efficient score function can also be seen to be equal to `˙_θ

0−c T_`˙

η0 for the

vectorcthat minimizes P_{θ0η0 ˙}`<sub>θ0−</sub>cT<sub>`˙</sub> η0‘2.

The notion of a “least favorable submodel” has been extended to semipara-metric models. Given a semiparasemipara-metric model of the type_”p_{θ; η}xθ∈2; η∈H•, the score function forθis defined, as usual, as the partial derivative with re-spect toθof the log density. Theefficient score functionforθis defined as

˜

`θ= ˙`θ−5`˙θ;

where5`minimizes the squared distanceP<sub>θη</sub>`₋k_‘2_{over all functionskin the}

closed linear span of the score functions for η. The inverse of the variance of

˜

`_θis the Cram´er–Rao bound for estimatingθin the presence ofη. A submodel

t_→p_t;η

t with ηθ=ηis defined to be least favorable atθ; η‘if

˜

`_θ= ∂ ∂t

t=θ

lnp_{t; η} t:

Since a projection 5`on the closed linear span of the nuisance scores is not necessarily a nuisance score itself, least favorable submodels may not always exist. (Problems seem to arise in particular at the maximum likelihood es-timator _{ ˆ}θ;η_ˆ‘, which may happen to be “on the boundary of the parameter set.”) However, in all our examples a least favorable submodel exists or can be approximated sufficiently closely.

In the case that the parameterψdoes not factorize naturally into a param-eter of interestθand a nuisance parameter, an efficient score function can be defined and calculated more elegantly in the following manner. (We use this in the example of doubly censored data withψ=Fthe distribution function andθ=Fg.) Assume that score functions for the full model can be written in the form

∂ ∂t

t=0

lnp_ψtx‘ =`_ψhx‘;

where h is a “direction” in which ψ_t approaches ψ, running through some Hilbert spaceH, and `_ψx H_→ L2Pψ‘ the “score operator.” [In the example

of doubly censored dataHis the set of all functions inL2F‘with meanFh

zero.] Furthermore, assume that the parameterθx9→Ris differentiable in the sense that, for someθ˙0∈H,

∂ ∂t

t=0

θ_ψ_{t‘ = Œ ˙}θ₀; h_ψ:

Then the Cram´er–Rao bound for estimatingθ_ψ_‘equals

sup

h

Œ ˙θ₀; h_2_ψ P_ψ`_ψh_‘2:

This supremum can be given a more concrete form by introducing the adjoint operator`∗

ψxL2Pψ‘ →H, which is characterized by the requirement P_ψ`<sub>ψ</sub>h<sub>‘</sub>g<sub>= Œ</sub>`_ψh; g_P

ψ = Œh; ` ∗

ψgψ for everyh∈H; g∈L2Pψ‘:

With this notation the efficient influence function for θis defined as the ele-mentg₀ of the closure of`_ψHsuch that

`∗_ψg_{0= ˙}θ₀:

Ifg₀can be written in the form `<sub>ψ</sub>h<sub>0</sub>for someh<sub>0∈</sub>H(it cannot always), and the “information operator”`∗_ψ`_ψis invertible, this readily yields the represen-tation

g0=`ψh0; h0= `∗ψ`ψ‘−1θ˙0:

(3.4)

In this formulation the variance of the efficient influence function g0 is the

Cram´er–Rao bound for estimating θ; the inverse of this variance could be defined as the information about θ. Thus g₀ corresponds to `˜<sub>θ</sub>/var`˜_θ in the case that ψ _{= }θ; η_‘ is partitioned. The direction h₀ is the least favorable direction in H; the derivative of the logarithm of a least favorable submodel

t_→p_ψ

t, intatt=0 is equal to`ψhwith

h₌ ` ∗

ψ`ψ‘−1θ˙0

Œ ˙θ₀;_`∗

ψ`ψ‘−1θ˙0ψ :

Note the similarity to (3.1). The Cram´er–Rao bound for the submodel in the least favorable direction gives the supremum in (3.3).

3.1. A general theorem. In this section we discuss our approach toward obtaining the asymptotic distribution of the likelihood ratio statistic, which is partly motivated by the preceding discussion. Of course, for any ψ˜ and any

˜

ψ0∈90= ”ψxθψ‘ =θ0•,

lrt_nθ_{0‘ =}2n_Pn ln lik_{ ˆ}ψ_{‘ −}ln lik_{ ˆ}ψ_0‘ (

≥2n_Pn ln lik_{ ˜}ψ_{‘ −}ln lik_{ ˆ}ψ_0‘;

≤2nPn ln lik_{ ˆ}ψ‘ −ln lik_{ ˜}ψ0‘

:

If both the upper and lower bounds converge to a chi-squared distribution on one degree of freedom, then this also holds for the likelihood ratio statistic. We use (3.2) as motivation to define suitable perturbationsψ˜ andψ˜₀ofψˆ and

ˆ

ψ₀. We defineψ˜₀by perturbingψˆ in a least favorable direction [so that in view of (3.2) it can be expected to resembleψˆ₀]; we defineψ˜ by perturbingψˆ₀ in a least favorable direction [so that in view of (3.2) it can be expected to resemble

ˆ

ψ]. Thus, the perturbations are constructed as elements of submodels that are approximately least favorable.

The following theorem gives a general framework for our approach. Denote

P_ψ0 by P0. We assume that as n tends to infinity, the maximum likelihood

estimator θˆ₌θ_{ ˆ}ψ_‘satisfies

√

n ˆθ−θ0‘ =

√

nPn`/˜ I˜+oP1‘; I˜=P0`˜2;

for a mean zero function `˜ with finite and positive variance I˜ under P0. In

all our examples the maximum likelihood estimator is asymptotically efficient and`/˜ I˜is the efficient influence function for estimatingθ.

Furthermore, we assume that there exist “approximately least favorable submodels.” For everytandψsuppose that there exists a curvet_→c_tψ_‘in

9 indexed by the parameter of interestt, and passing throughψatt₌θ_ψ_‘. Technically this means that

θ_c_tψ_{‘‘ =}t and c_tψ_{‘Žt=θψ‘=}ψ:

(3.6)

(The proof below uses these curves only for t close to θ₀ and for ψ _{= ˆ}ψ or

ψ_{= ˆ}ψ₀.) The curvet_→c_tψ_‘should be approximately least favorable in that the submodel

t_→`_x_yt; ψ_{‘ =}ln lik_c_tψ_‘; x_‘

(3.7)

should be twice continuously differentiable for everyx, with derivatives`˙and

¨

`satisfying

−Pn`¨<sub>·y ˜</sub>θ;ψ˜<sub>‘ →P</sub>P<sub>0</sub>`˜2 for any randomθ˜_→Pθ₀; ψ˜ _→Pψ₀;

(3.8)

√

n_{Pn ˙}`<sub>·y</sub>θ<sub>0</sub>;ψˆ<sub>0‘ − ˜</sub>`_{‘ →P}0:

(3.9)

The idea is to construct the submodel such that the first derivative`˙_·; θ0; ψ0‘

is equal to the efficient score function`˜, whence the expectation of its second derivative should be minus the efficient information forθ.

The preceding conditions presume a topology on the set9, and we assume that the maximum likelihood estimators are consistent with respect to this topology.

Theorem 3.1. Suppose that the mapst_→`_x_yt; ψ_‘are twice continuously differentiable and satisfy (3.5)–(3.9), and suppose thatψˆ andψˆ₀are consistent. Thenlrt_nθ_0‘ χ2

1.

Proof. Since, by (3.6), ψˆ ₌c_{θˆ ˆ}ψ_‘,

lrt_nθ_{0‘ =}2n_Pnln lik_{ ˆ}ψ_{‘ −}ln lik_{ ˆ}ψ_0‘

≤2n_Pnln lik_c_{θˆ ˆ}ψ_{‘‘ −}ln lik_c_θ

0 ˆψ‘‘

=2n_{Pn− ˙}`<sub>·y ˆ</sub>θ;ψˆ<sub>‘</sub>θ<sub>0− ˆ</sub>θ<sub>‘ −</sub>1/2`¨_{·y ˜}θ;ψˆ_‘θ_{0− ˆ}θ_‘2_;

Since, by (3.6),ψˆ0=cθ0 ˆψ0‘,

lrt_nθ0‘ =2nPn

ln lik_{ ˆ}ψ‘ −ln lik_{ ˆ}ψ0‘

≥2n_Pnln lik_c_{θˆ ˆ}ψ0‘‘ −ln likcθ0 ˆψ0‘‘

=2n_Pn`˙<sub>·y</sub>θ<sub>0</sub>;ψˆ<sub>0‘ ˆ</sub>θ<sub>−</sub>θ<sub>0‘ +</sub>n<sub>Pn</sub>`¨_{·y ˜}θ;ψˆ_0‘θ_{0− ˆ}θ_‘2;

for someθ˜betweenθ₀andθˆ. Apply (3.9) to get that the first term on the right is equal to 2√n_{ ˆ}θ₋θ_0‘√n_Pn`˜<sub>+</sub>o<sub>P</sub>1<sub>‘</sub> and apply (3.8) to get that the second term is<sub>−</sub>P<sub>0</sub>`˜2_{n ˆθ−θ}

0‘2+oP1‘. An application of (3.5) shows that the

right-hand side converges in distribution to a chi-squared distribution.

The combination of the preceding two paragraphs yields the theorem. 2

We note that it is sufficient that the conditions of the theorem are true “with probability tending to 1.” Similarly, it suffices that the “paths”_”c_{t ˆ}ψ_‘xt_∈U_•

and_”c_{t ˆ}ψ0‘xt∈U•belong to the parameter set with probability tending to 1

for a fixed neighborhoodU of θ0 (not depending onn). We keep this in mind

when discussing our examples.

Example(Doubly censored data). The theorems are applied with the sub-modelc_tψ_{‘ =}F_tθ; F_‘, where

F_tθ; F_{‘ =}F_{+ }θ₋t_‘Z·_g∗₋Fg∗_‘dF/_−I_F‘;

I_F=Z g_g∗₋Fg∗_‘dF:

(3.10)

(Note thatθ₌Fgby definition.) The functiong∗ _{is the least favorable} direc-tion in theF-space at the true distributionF₀for estimatingFgand is defined by (4.1). It will be shown to be bounded and of bounded variation. The expres-sion I_F0 is the inverse of the information I˜ and is positive. The expression

F_tθ; F_‘does not truly define a probability distribution for everyt₋θandF, although it always has total (signed) mass 1. However, in view of the bound-edness ofg∗_{, ift−θ,Fgg}∗_−F0gg∗_,Fg∗_−F0g∗ _{andFg−F0gare sufficiently} close to zero, thenF_tθ; η‘has a positive density 1_{+ }θ−t‘g∗_−Fg∗_‘/I

Fwith

respect toFand hence defines a probability distribution.

Example (Cox regression for current status data). The theorems are ap-plied withc_tψ_{‘ = }t; 3_tθ; 3_‘‘, where

3_tθ; 3_{‘ =}3_{+ }θ₋t_‘φ_3_‘h∗∗_◦3−1 0 ◦3:

(3.11)

Here the function3₀h∗∗_{is the least favorable direction for estimatingθin the} 3-space at the true parameter_θ₀; 3_0‘and is defined by (5.1), andφ_{x ’}0; M_{“ →}

’0;_∞‘ is a fixed function such that φ_y_{‘ =} y on the interval_’30σ‘; 30τ‘“,

such that the function y→ φy‘/y is Lipschitz and such thatφy‘ ≤cy∧ M−y‘‘for a sufficiently large constantcspecified below [and depending on

θ0; 30‘ only]. [By our assumption that ’30σ‘; 30τ‘“ ⊂ 0; M‘ such a

least favorable direction, but its definition is somewhat complicated in order to ensure that 3_tθ; 3_‘ really defines a cumulative hazard function within our parameter space, at least for tthat are sufficiently close to θ. First, the construction using h∗∗_◦3−1

0 ◦3, rather than h∗∗ [taken from Huang (1996)],

ensures that the perturbation that is added to3is absolutely continuous with respect to 3; otherwise3_tθ; 3_‘ would not be a nondecreasing function. Sec-ond, the functionφ“truncates” the values of the perturbed hazard function to

’0; M_“.

A precise proof that3_tθ; 3_‘is a parameter is as follows. Since the function

φ is bounded and Lipschitz and by assumption, h∗∗_{◦ 3}−1

0 is bounded and

Lipschitz, so is their product and hence, foru_≤v and_Žθ₋t_Ž< ε,

3_tθ; 3_‘v_{‘ −}3_tθ; 3_‘u_{‘ ≥} 3_v_{‘ −}3_u_‘ 1₋ε_φh∗∗◦3−₀1_Lipschitz:

For sufficiently smallεthe right-hand side is nonnegative. Next, for_Žθ₋t_Ž< ε,

3_tθ; 3_{‘ ≤}3₊εφ_3_‘h∗∗_∞:

This is certainly bounded above byM(on_’0; τ_“) ifφ_y_{‘ ≤ }M₋y_‘/_ε_h∗∗_ ∞‘ for all 0_≤y_≤M. Finally,3_tθ; 3_‘can be seen to be nonnegative on_’σ; τ_“by the condition thatφ_y_{‘ ≤}cy.

Example (Gamma frailty). The theorems are applied with c_tψ_{‘ =}

t; 3_tθ; 3_‘‘, where

3_tθ; 3_{‘ =}3_{+ }θ₋t_‘Z·k∗d3:

(3.12)

The function k∗ _{is the least favorable direction in the 3-space at the true} parameter_θ0; 30‘ and is defined by (6.1). This function will be shown to be

bounded, so that the density 1_+θ−t‘k∗_of3

tθ; η‘with respect to3is positive

for sufficiently small _Žθ−tŽ. In that case 3_tθ; η‘ defines a nondecreasing function and is a true parameter of the model.

Example (Mixture model). The theorems are applied with c_tψ‘ = t; F_tθ; F‘‘, where

F_tθ; F_‘B_{‘ =}F

B

1₊θ−t 2θ

₋1 :

(3.13)

This will be shown to be an exact least favorable submodel at Fand is well defined whenever_Žθ₋t_Ž<2θ.

Having defined suitable submodels, we next need to check the technical conditions of Theorem 3.1. Regarding condition (3.8), we note that

¨

`_·yt; ψ_{‘ =} ∂ 2_likc

tψ‘‘/∂t2

lik_c_tψ‘‘ − ˙`

2

In one of our four examples we use paths such that the maps t → lik_c_t‘

are linear in which case the first term on the right-hand side is zero. In the other examples the first term has the form _∂2_pt/∂t2_{‘/pt, which is a mean}

zero function underp_t and can be shown to give a negligible contribution:

Pn∂2likctψ‘‘/∂t2

lik_c_tψ_‘‘

t= ˜θ

→P0 for everyθ˜→Pθ0:

(3.14)

This could be proved by classical arguments or with the help of the mod-ern Glivenko–Cantelli theorem: if the functions are contained in a Glivenko– Cantelli class, the empirical measure can be replaced by the true measure.

Conditions (3.8) and (3.9) can often be checked with help of the following lemma, which uses concepts from the theory of empirical processes. See, for instance, van der Vaart and Wellner (1996) for a review and methods to check the conditions.

Lemma 3.2. Suppose that there exist neighborhoods U of θ0 and V of ψ0

such that the class of functions`˙·yt; ψ‘x t∈ U; ψ∈V isP0-Donsker with

square-integrable envelope function. Furthermore, suppose that `˙_x_yt; ψ_{‘ →}

˜

`_x_‘ for P₀-almost every x, as t _→ θ₀ and ψ _→ ψ₀. Then for all random

sequencesθ˜_nandψ˜_nthat converge in probability to θ₀andψ₀we have

Pn`˙2 <sub>·y ˜</sub>θ<sub>n</sub>;ψ˜<sub>n→</sub>PP0`˜2;

√

nPn−P0‘ ˙`·y ˜θn;ψ˜n‘ − ˜`

→P0:

Proof. Assume without loss of generality thatθ˜_nandψ˜_ntake their values inUandV, respectively.

By Theorem 4.6 of Gin´e and Zinn (1986) or Lemma 2.10.14 of van der Vaart and Wellner (1996) the class of squaresf2of functionsfranging over a Donsker class with square integrable envelope is Glivenko–Cantelli. It follows that

sup

t∈U; ψ∈V

_Pn−P_{0‘ ˙}`2_·yt; ψ_‘→P0:

Thus in the first statement the empirical measure may be replaced by the true underlying measure. By assumption `˜x‘ = ˙`xyθ0; ψ0‘ almost surely under P0. Next the result follows by the dominated convergence theorem.

For the second statement define a stochastic process

G_n=√n_Pn−P_{0‘ ˙}`<sub>·y</sub>t; ψ<sub>‘ − ˜</sub>`_xt_∈U; ψ_∈V :

The Donsker assumption (and the square integrability of the envelope func-tion) entails that the sequenceG_n is asymptotically uniformly continuous in probability, that is, for everyε >0,

lim

δ↓0lim supn→∞

P∗ _sup

ρt;ψ‘;t0_;ψ0_‘‘<δ

whereρis the semimetric given by its square

ρ2 _t; ψ_‘;_t0; ψ0_‘=P₀ `˙<sub>·y</sub>t; ψ<sub>‘ − ˙</sub>`_·yt0; ψ0_‘2:

By dominated convergence and consistency of_{ ˜}θn;ψ˜n‘we have

ρ2 _{ ˜}θ;ψ˜_‘;_θ₀; ψ_0‘→P0:

Conclude that the sequenceG_n ˜θ_n;ψ˜_n‘ −G_nθ0; ψ0‘converges in probability

to zero. This is the second assertion of the lemma. 2

Once (3.14) is verified and if the preceding lemma is applicable, the condi-tions of Theorem 3.1 effectively reduce to the condition

√

nP₀`˙_·yθ₀;ψˆ_{0‘ →P}0:

(3.15)

Whereas the conditions of the lemma can be viewed as regularity conditions, this is a structural condition. An “unbiasedness” condition of this type may be expected in view of results of Klaassen (1987), who shows that if θ can be estimated efficiently, then there must exist (consistent) estimators`ˆof the efficient score function such that√nP<sub>0</sub>`ˆ_→P0. The preceding display requires that the plug-in estimator`˙_·yθ₀;ψˆ_0‘has the latter property.

A similar condition (with ψˆ instead of ψˆ₀) also shows up in proofs of the asymptotic normality of the maximum likelihood estimatorθˆ. [See, e.g., Huang (1996) or van der Vaart (1996).]

At first, condition (3.15) appears to require a rate of convergence of ψˆ0.

This is not true, as in many cases `˙·yt; ψ‘ is of a special form. For instance, in semiparametric models in which the density is convex linear in the nui-sance parameter, the efficient score function for θ is unbiased in the sense that P<sub>θη</sub>`˜_θθ; η_{0‘ =}0 for anyθ, ηandη₀. In our mixture model example we construct the submodel t _→ c_tψ_‘ in such a way that the derivative is ex-actly the efficient score function, and the unbiasedness condition is trivially satisfied.

In the worst situation (3.15) should not require more than ano_Pn−1/4_‘-rate

for the nuisance parameter. For instance, if ψ _{= }θ; η_‘ and `˜_·yθ₀;ψˆ_0‘is the efficient score function, then the expression in (3.15) is equal to

√

n_P₀₋P_ψˆ

0‘ ˜`·yθ0;ψˆ0‘ ≈ −

√

nP_ψˆ

0`˜·yθ0;ψˆ0‘`η0ˆ  ˆη−η0‘ +

√

nO _{ ˆ}η₋η_02:

Here the first term should vanish, because the efficient score function forθis orthogonal to all scores for the nuisance parameter.

4. Doubly censored data. The path defined by dF_t = 1₊th‘dFfor a bounded function withFh=0 yields a score function of the form

`_Fh_u; d_{‘ =} R

’0; u“h dF

Fu‘ I”d=1• +hu‘I”d=2• +

R

u;∞‘h dF

For g and h bounded, we have that PF`Fh‘`Fg‘ = Œ`Fh‘; `Fg‘PF =

Œh; `∗<sub>`</sub>

FgF = Fh `∗`Fg‘‘. So we may use Fubini’s theorem to derive the

adjoint operator`∗_xL

2PF‘ →L2F‘, `∗g_s_{‘ =}Z

’s;∞‘gu;1‘dGLu‘ +gs;2‘GL−GR‘s−‘ + Z

’0; s‘gu;3‘dGRu‘: Thus the information operator takes the form

`∗<sub>`</sub>

Fhs‘ = Z

’s;∞‘ R

’0; z“h dF

F_z_‘ dGLz‘ +hs‘GL−GR‘s−‘

+Z

’0; s‘ R

y; τ“h dF

1₋Fy‘ dGRy‘:

Since the functiongused to define the null hypothesisFg₌θ₀is assumed to be bounded and of bounded variation, part (i) of Lemma A.2 shows that the function

g∗ _{= `</sub>∗<sub>`} F0‘

−1_g

(4.1)

is well defined, bounded and of bounded variation. We use this function to define the (approximately) least favorable submodel (3.10).

In Lemma A.3, we prove consistency ofFˆ₀and that

 ˆF_{− ˆ}F_0‘h_{‘ =}F₀ h_g∗_−F 0g∗‘

 ˆθ₋θ_0‘/F₀ g_g∗_−F 0g∗‘

+o_Pn−1/2_‘

uniformly for uniformly bounded h of uniformly bounded variation. Since the asymptotic variance of √n_{ ˆ}θ₋θ_0‘ is F_0g_g∗ _−F

0g∗‘‘, this confirms

the intuition expressed in Section 3. In the verification of Lemma A.3(ii) we show that _{ ˆ}F₋F_{0∞ =} O_Pn−1/2_{‘ and  ˆF}

0−F0∞ = OPn−1/2‘ and that

 ˆθ−θ0‘ =Pn`F0g ∗_−F

0g∗‘ +oPn−1/2‘so`/˜ I˜=`F0g ∗_−F

0g∗‘.

Recalling (3.7) and (3.10), we have

˙

`u; dyt; F‘ =<sub>1</sub> `Fg∗−Fg∗‘u; d‘/IF

+ t₋Fg_‘`<sub>F</sub>g∗<sub>−Fg</sub>∗<sub>‘u; d‘/IF</sub>; IF=Fgg∗−Fg∗‘: Given a bounded, monotone functionhthe function`_Fhis composed of three bounded and monotone functions, with the same uniform bound. Since g∗ is bounded and of bounded variation, it follows that the class of functions

`_Fg∗ _{with F ranging over all distributions on ’σ; τ“ consists of uniformly} bounded functions of uniformly bounded variation, hence is a Donsker class. For _F₋F_0∞ sufficiently small we have that I_F is close to I_F0, Fg is close to F₀g and Fg∗ _{is close to F}

0g∗. Given Donsker classes F1; : : : ;Fk

and a Lipschitz function φ_{x R}k _{→ R}, a uniformly bounded class of functions

x→φ f1x‘; : : : ; fkx‘

y f_i∈Fi; i=1; : : : ; k, is Donsker by Theorem 2.10.6

of van der Vaart and Wellner (1996). It follows that the class of functions

˙

`u; dyt; F‘ with t sufficiently close toθ0 =F0g andF−F0∞ sufficiently small is Donsker. As _t; F‘ → θ0; F0‘ these functions converge a.s.-PF0 to

˜

` = ˙`·yθ0; F0‘. Thus the conditions of Lemma 3.2 are satisfied. Note that

tis_{− ˙}`u; dyt; F‘‘2_{. For the application of our Theorem 3.1 it suffices to show}

that

√

nP₀`_Fˆ 0g

∗_{− ˆF} 0g∗‘ =

√

n_P₀₋P_Fˆ 0‘`Fˆ0g

∗₌√_nF

0− ˆF0‘`∗`Fˆ0g ∗_→

P0:

Since_{ ˆ}F₀₋F_0‘g₌0 the absolute value of this expression is equal to, in view of the definition ofg∗ _{and that the support ofFˆ}

0 is contained in’σ; τ“,

√

nF0− ˆF0‘ `∗`Fˆ0g ∗_{−`</sub>∗<sub>`}

F0g∗≤2

√

n ˆF0−F0∞ `∗`_Fˆ

0g ∗_{−`</sub>∗<sub>`}

F0g∗

BV;

where_{ŽŽ · ŽŽBV}is the sum of the supremum and total variation norms. The first term on the right-hand side is bounded in probability; the second converges to zero in probability by Lemma A.2(ii).

5. Regression for current status data. In this model the score function forθtakes the form

`_θθ; 3_‘x_{‘ =}z3_y_‘Q_x_yθ; 3_‘;

for the functionQ_x_yθ; 3_‘given by

Q_x_yθ; 3_{‘ =}eθz

δ exp−expθz‘3y‘‘

1₋exp_−exp_θz_‘3_y_‘‘ − 1−δ‘

:

Inserting into the log likelihood a submodel t → 3_t such that hy‘ = −∂/∂tŽt=0 3ty‘ exists for every y, and differentiating at t = 0 we obtain a

score function for3of the form

`_3θ; 3_‘h_x_{‘ =}h_y_‘Q_x_yθ; 3_‘:

For every nondecreasing, nonnegative functionhthe submodel3_{t =}3₊this well defined fort_≥0 and yields a (one-sided) derivative hat t₌0. Thus the preceding display gives a (one-sided) score for3at least for allhof this type. The linear span of these functions contains`<sub>3</sub>hfor all bounded functionshof bounded variation. The efficient score function forθis defined as`_θ−`₃h∗ _for h∗ _{minimizing the distanceP}

θ3`θ−`3h‘2. In view of the similar structure of

the scores for θand 3this is a weighted least squares problem with weight functionQ_y; δ; z_yθ; 3_‘. The solution at the true parameters is given by

h∗_Y_{‘ =}3_0Y_‘h∗∗_Y_{‘ =}3_0Y_‘Eθ030 ZQ 2_Xyθ

0; 30‘ŽY

E_θ030 Q2_Xyθ

0; 30‘ŽY :

(5.1)

As the formula shows (and as follows from the nature of the minimization problem) the functionh∗∗_{is unique only up to null sets for the distribution of} Y. However, it is an assumption that (under the true parameters) there exists a version of the conditional expectation that is differentiable with bounded derivative. Following Huang (1996) this version is used to define the least favorable submodels (3.11). By the assumption that P₀var_Z_ŽY_‘ > 0, the efficient score function `˜ <sub>=</sub> `_θθ₀; 3_{0‘ −}`_3θ₀; 3_0‘h∗_{, the difference between} the θ-score and its projection, is nonzero, whence the efficient information I˜

The consistency of _{ ˆ}θ;3ˆ‘ and3ˆ0 can be proved by a standard consistency

proof, where we may start from the inequality _Pnln_lik_{ ˆ}ψ_{‘ +}lik_ψ_{0‘‘ ≥} Pnln 2lik_ψ_0‘, rather than from the more obvious inequality _Pnln lik_{ ˆ}ψ_{‘ ≥} Pnln lik_ψ_0‘. (The latter inequality implies the first by the concavity of ln.) See, for instance, the consistency proof in Huang and Wellner (1995), or also the Appendix. The identifiability of the parameters is ensured by the assumption that P₀var_Z_ŽY_‘>0. More precisely, it can be shown under our conditions that on a set of probability 1, 3ˆ and 3ˆ₀ converge to3₀ uniformly on the interval _’σ ₊ε; τ ₋ε_“, for every ε > 0. Of course, the estimators and the true distribution are not identifiable outside the interval _’σ; τ_“. The consistency of the estimators atσ andτ seems dubious, even though this is used by Huang (1996) in some of his proofs. In the Appendix, we show that the asymptotic normality and efficiency (3.5) ofθˆremains valid even without the uniform consistency on _’σ; τ“. We also show that both R_στ_{ ˆ}3−30‘2y‘dy

[as asserted by Huang (1996)] and R_στ_{ ˆ}30−30‘2y‘dyconverge to zero at the

rateO_Pn−2/3_{‘. We shall use this to verify condition (3.15).}

In view of (3.11), we have

˙

`xyt; 3; θ‘ =

z−₃φ3‘y‘

tθ; 3‘y‘

h∗∗◦3−₀1◦3y‘

3_tθ; 3‘y‘Q xyt; 3_tθ; 3‘:

For_t; 3; θ_‘tending to_θ₀; 3₀; θ_0‘this function converges almost everywhere to its value at _θ₀; 3₀; θ_0‘, which, by construction, is the efficient score func-tion `˜for θ at <sub></sub>θ<sub>0</sub>; 3<sub>0‘</sub>. Furthermore, the class of functions `˙_x_yt; 3; θ_‘ with

t; θ_‘ varying over a small neighborhood of _θ₀; θ_0‘ and with3 ranging over all nondecreasing cadlag functions with range in _’0; M_“ can be seen to be a Donsker class by repeated application of preservation properties for Donsker classes [cf. van der Vaart and Wellner (1996), Chapter 2.11]. Note here that, since the functionu_→ue−u_/1−e−u_{‘is bounded and Lipschitz on’0;∞‘, we}

can write

3_y_‘Q_x_yt; 3_{‘ =}ψ etz_{; 3y‘}

for a function ψ that is bounded and Lipschitz in its two arguments. Thus, since the classes of functionsz→etz_{andy→3y‘are Donsker, so is the class}

of functions x→ 3y‘Qxyt; 3‘. Next, since the function φy‘/yis bounded and Lipschitz, andh∗_◦3−1

0 is assumed bounded and Lipschitz, φ_3_‘

3_tθ; 3_‘ =

φ_3_‘/3

1_{+ }θ₋t_‘φ_3_‘/3 h∗∗_◦3−1 0 ◦3

=χ_θ₋t; 3_‘

for a functionχthat is bounded and Lipschitz on an appropriate domain. Next, the product of the two classes of functions in the preceding displays, which are both uniformly bounded and Donsker, is Donsker, and so on.

these two conditions we compute that

∂2lik t; 3tθ; 3‘

/∂t2

lik t; 3_tθ; 3_‘ =Q ·yt; 3tθ; 3‘‘

×hz23_tθ; 3_{‘ −}2φ_3_‘h∗∗◦3−₀1◦3

−etz z3_tθ; 3‘ −φ3‘h∗∗◦3−₀1◦32i:

By the same arguments as before these functions are in a Donsker class, hence certainly in a Glivenko–Cantelli class. Thus the empirical measure_Pn in (3.14) can be replaced by the true measure P₀. As _t; θ; 3_‘ converges to

θ₀; θ₀; 3_0‘the functions in the preceding display converge almost everywhere to _∂2_p

t; 3tθ0; 30‘/∂t 2_‘/p

t; 3tθ0; 30‘ evaluated att=θ0. This has mean zero. We

can conclude the proof of (3.14) by the dominated convergence theorem. Abbreviating`˙<sub>·y</sub>θ<sub>0</sub>; 3; θ<sub>0‘</sub>to`˙_3_‘, we can rewrite the expectation in (3.15) in the form

P0`˙ ˆ30‘ = P0−Pθ0;3ˆ0‘ ˙`30‘ + P0−Pθ0;3ˆ0‘ ˙` ˆ30‘ − ˙`30‘

:

(5.2)

We shall show that both terms on the right-hand side are of the order

O_Pn−2/3_{‘ and hence certainly o}

Pn−1/2‘. Since `˙30‘ is the efficient score

function for θand hence is orthogonal to every 3-score, the first term can be rewritten as

P0`˙30‘

p0−pθ0;3ˆ0‘/p0−`3θ0; 30‘30− ˆ30‘

:

The second term in square brackets is exactly the linear approximation in

3_{0− ˆ}3₀ of the first. Taking the Taylor expansion one term further shows that the term in square brackets is bounded by a multiple of_30− ˆ30‘2 and hence

the display is bounded by a multiple of P_03_{0− ˆ}3_0‘2_{. The second term in}

(5.2) can be bounded similarly, since both3_→p_θ

0; 3 and3→ ˙`θ0; 3; θ0‘are

uniformly Lipschitz functions.

6. Gamma frailty. The natural log of (2.1) is given by

ln lik_θ; 3; X‘ =Zτ

0

ln 1₊θNu−‘dNu‘

− 1₊θN_τ_‘θ−1ln

1₊θZτ 0 Yd3

+Zτ

0 ln Yu‘13u‘

dN_u_‘;

where if θ ₌ 0, θ−1_ln1+θRτ

0Y d3‘ is set to its limit, Rτ

0 Y d3. The score

function forθis

`_θθ; 3_{‘ =}Zτ 0

N_u_−‘

1₊θNu−‘dNu‘

−θ−1Zτ 0 Y d3

1₊θNτ‘

1₊θR₀τY d3+θ −2_ln

1₊θZτ 0 Y d3

where, in the case that θ=0 the last two terms above are replaced by their limit,

−N_τ_‘Zτ

0 Y d3+ 1 2

Zτ

0 Y d3 2

:

The path defined by d3_{t = }1₊th_2‘d3for a bounded function,h₂, yields a score function for3of the form,

`_3θ; 3_‘Z·h₂d3₁

=Zτ

0 h2dN−

1₊θNτ‘

1₊θR₀τY d3 Zτ

0 Yh2d31:

Also define three second derivatives. The second derivative`<sub>θθ</sub>θ; 3<sub>‘</sub> is ob-tained by simply differentiating`_θθ; 3_‘with respect toθ. For boundedh₂and

g₂, the remaining second derivatives are defined as

`_θ3θ; 3_‘Z·h₂d3₁

= −

Rτ

0 Yh2d31

1₊θR₀τY d3

×

N_τ_{‘ −} 1+θNτ‘

1₊θR₀τY d3 Zτ

0 Y d3

`₃₃θ; 3‘Z·h2d31; Z·

g2d32

= 1+θNτ‘

1₊θR₀τY d3_‘2 Z·

h2Y d31 Z·

g2Y d32

− 1+θNτ‘

1₊θR₀τY d3 Zτ

0 h2g2Y d32:

Finally, letting BV_’0; τ_“denote the functionsh_{x ’}0; τ_{“ →R}which are bounded and of bounded variation, equipped with the norm _{ŽŽ · ŽŽBV = ŽŽ · ŽŽ∞+ ŽŽ · ŽŽvar}, define operators σ₃₃x BV_’0; τ“ → BV_’0; τ“ and σ = σ1; σ2‘x R×BV’0; τ“ → R×BV_’0; τ“by

σ_33’h_2“u_{‘ =}P₀

−Y_u_‘θ₀ 1+θ0Nτ‘

1₊θ0 Rτ

0Y d30‘2 Z·

h₂Y d3₀

+h2u‘P0

Yu‘₁ 1+θNτ‘ +θR₀τY d3₀

;

σ1h1; h2‘ = −h1P0`θθθ0; 30‘ −P0`θ3θ0; 30‘ Z ·

h2d30

σ2h1; h2‘u‘ =h1P0

_Y

u_‘

1₊θ0 Rτ

0Y d30

Nτ‘ −₁ 1+θ0Nτ‘ +θ0

Rτ 0Y d30

Zτ

0 Y d30

+σ₃₃’h2“:

operator`∗<sub>3</sub>`3connected to the nuisance parameter3; that is,

Œg₂; σ_33’h_2“3=

`_3θ₀; 3_0‘Z ·g₂d3₀

; `_3θ₀; 3_0‘Z·h₂d3₀

P0 :

This follows from the identity

Zτ

0 g2σ33h‘d30= −P0`33θ0; 30‘ Z·

h₂d3₀;Z·g₂d3₀

=P₀`_3θ₀; 3_0‘Z·h₂d3₀

`_3θ₀; 3_0‘Z·g₂d3₀

:

Similarly, the operatorσ is the information operator`∗<sub>ψ</sub>`_ψψ0‘for the full

pa-rameterψ= θ; 3‘of the model, for which the score function can be written as

`<sub>ψ</sub>ψ‘h1; h2‘and is defined ash1`θθ; 3‘ +`3θ; 3‘’ R_·

h2d3“. The parameter

of interestθ_ψ_{‘ =}θhas derivativeθ˙0= 1;0‘, since d

dt

t=0

θθ+th1; 3t‘ =h1=

h1; h2‘;1;0‘

R×3:

Thus by the general theory [see (3.4)] we can find the influence function for estimatingθ, lettingσ_{˜ =  ˜}σ₁;σ_˜2‘be the inverse ofσ, as

`<sub>ψ</sub>ψ<sub>0‘</sub>`∗_ψ`<sub>ψ</sub>ψ<sub>0‘‘</sub>−1<sub></sub>1;0<sub>‘ = ˜</sub>σ<sub>1</sub>1;0<sub>‘</sub>`_θθ₀; 3_{0‘ +}`_3θ₀; 3_0‘Z·σ_˜21;0_‘d3₀

:

The following lemma shows that this function is well defined. Denote the subset of BV_’0; τ“, with norm bounded above byp, by BV_p’0; τ“.

Lemma 6.1. Under the conditions of Theorem 2.3 the following hold:

(i) σ_33xBV_’0; τ_{“ →}BV_’0; τ_“is continuously invertible with inverseσ_˜33;

(ii) σ_xR×BV_’0; τ_{“ →R×}BV_’0; τ_“is continuously invertible with inverseσ_˜;

(iii) √n_{ ˆ}3₋3_0∞ isO_P1_‘;

(iv) √n_{ ˆ}3_{0·‘ −}3_0·‘‘ Z_·‘ on l∞_BV

p’0; τ“‘, where Z is a tight

Gauss-ian process with mean zero and covarGauss-iance process covar_Z_h_2‘; Z_g_{2‘‘ =} Rτ