FIC for comparing two samples

The FIC schemes we have considered so far may be used in dierent situations where there is a single sample only. In some situations one may be interested in a focus parameter that depends on several samples. The classical example of such a situation is the dierence between the means of two samples. Such a quantity may clearly be estimated both based on parametrics and nonparametrics. As a consequence, model selection techniques should be used to select which model one should base further inference on. In this section we discuss information criteria to select between models in this situation. We will investigate focused model selections for two of the most natural types of focus parameters comparing two samples. Firstly we consider focus parameters which is a dierence of two focus parameters, and then we consider a focus parameter which is a product of two focus parameters. We will also restrict the research to the iid situation considered in chapter 3.

In mathematical terms we assume the following situation: Y1, . . . , Y_nare iid random variable with a common distribution whose cdf is given by G1, and denoted the rst sample. Similarly X1, . . . , Xnare iid random variable with a common distribution whose cdf is given by G2, and is denoted the second sample. The two samples are also assumed to be independent of each other. In addition we assume that both samples and any focus parameters dened on the samples satisfy the regularity conditions in assumption 3.1.1.

5.3.1 Dierence of two focus parameters

Consider now the situation where the focus parameter is the dierence of two individual focus parameters where each of them depends only on one of the samples, and not both on the same sample. In mathematical terms we write this as a functional of the following form

µ = µ(G1, G2) = µ1(G1) − µ2(G2), (5.5) where µ1 and µ2 are functionals dened only on respectively the rst and second sample. For this type of focus parameters, we consider the following type of estimators:

µ = µ( bb G₁, bG₂) = µ₁( bG₁) − µ₂( bG₂),

for some estimatorsGb₁, bG₂ of the cdfs G1, G₂. Still working in the parametric vs. nonparamet-ric world, these estimates will typically consist ofGb_n and F_θbn for the two data sets. As usual we use the mse as a measure of the uncertainty of an estimator like in equation (5.5). Note that since the samples are independent the covariance between µ1( bG₁) and µ2( bG₂)is zero. For

5.3. FIC FOR COMPARING TWO SAMPLES 91 our convenience, let us write VarGi for VarGi(µ_i( bG_i))and biasGi for biasGi(µ₂( bG_i))for i = 1, 2 in addition to CovG1,G2 for CovG1,G2(µ1( bG1), µ2( bG2)). We then get

mse(µ( bG1, bG2)) = biasG1,G2(µ( bG1, bG2))²+ VarG1,G2(µ( bG1, bG2))

= (bias_G1− bias_G2)²+ Var_G1+ Var_G2 − 2Cov_G1,G2

= bias²_G1 + bias²_G2− 2bias_G1biasG2 + VarG1 + VarG2

= mse(µ1( bG1)) + mse(µ2( bG2)) − 2biasG1biasG2,

We do actually get a mse-formula that adds the two marginal mean squared errors, and sub-tracts a correction term. The correction term reduces the error in the case where the bias of the estimators has the same sign and increases the error when they have dierent signs.

To estimate this quantity we need estimators of the unsquared biases for the estimators of both µ1 and µ2. This is however directly provided byµb1−µb1,np andµb2−µb2,np sinceµb1,np and µb_2,np are unbiased estimators under the usual conditions. Hence, the natural estimator for this mse is given by

mse(µ( bd G1, bG2)) =mse(µd 1( bG1)) +mse(µd 2( bG2)) − 2(bµ1−bµ1,np)(µb2−µb2,np). (5.6) For the simplest case of just one parametric model, which are used for both samples, we get the following estimators:

1. Nonparametric + nonparametric: µbnp,np=µb1,np−µb2,np. 2. Nonparametric + parametric: bµ_pm,np =µb_1,pm−µb_2,np. 3. Parametric + nonparametric: µb_np,pm=bµ_1,np−µb_2,pm. 4. Parametric + parametric: µbpm,pm=µb1,pm−µb2,pm.

For these estimators, equation (5.6) motivates the following mse estimators FIC(µbnp,np) = 1

nVb1,np+ 1 mVb2,np, FIC(µb_pm,np) = (bµ_1,pm−µb_1,np)²− 1

nVb_1,np+ 21

nVb_1,pm,np+ 1 mVb_2,np, FIC(µb_np,pm) = 1

nVb_1,np+ (µb_2,pm−µb_2,np)²− 1

mVb_2,np+ 21

mVb_2,pm,np, FIC(µb_pm,pm) = (bµ_1,pm−µb_1,np)²− 1

nVb_1,np+ 21

nVb_1,pm,np+ (µb_2,pm−µb_2,np)²− 1 mVb_2,np + 21

mVb2,pm,np− 2(µb1,pm−µb1,np)(µb2,pm−µb2,np).

Note that the correction term is nonzero only in the last estimator consisting of only parametric estimators. As usual, the FIC scheme chooses the estimator with the smallest FIC value.

5.3.2 Product of two focus parameters

The form of the focus parameter in the above section is maybe the most useful one. However, in some cases one might want to take a look at focus parameters on a slightly dierent form.

Consider a focus parameter on the following multiplicative form:

µ = µ(G1, G2) = µ1(G1)µ2(G2),

where µ1 and µ2 are functionals dened on respectively the rst and second sample. Like in the previous section we derive the mse of this focus parameter for a general estimator where Gb₁ and Gb₂ are inserted to estimate respectively G1 and G2. When denoting the expectation of the estimators µ1( bG₁) and µ2( bG₂) by respectively EG1 and EG2, and otherwise using the notation of the previous section, we get

mse(µ( bG1, bG2)) = biasG1,G2(µ( bG1, bG2))²+ VarG1,G2(µ( bG1, bG2))

= (E_G1E_G2− µ_1,trueµ2,true)²+ µ²_1,trueVar_G2+ µ²_2,trueVar_G1 + Var_G2Var_G2

= E_G²₁E_G²₂ + µ²_1,trueµ²_2,true− 2E_G1E_G2µ_1,trueµ_2,true + µ²_1,trueVarG2+ µ²_2,trueVarG1 + VarG2VarG2

= mse(µ₁( bG₁))mse(µ₂( bG₂)) + E_G²₁E_G²₂

− 2E_G1E_G2µ1,trueµ2,true.

Using the same estimators as earlier, we get the following natural mse estimator:

mse(µ( bd G₁, bG₂)) =mse(µd ₁( bG₁))mse(µd ₂( bG₂)) + µ₁( bG₁)²µ₂( bG₂)²

− 2µ₁( bG₁)µ₂( bG₂)µb_1,npµb_2,np. (5.7) For the simplest case of just one parametric model, which we apply to both samples, we get four natural estimators for µ:

1. Nonparametric + nonparametric: µbnp,np=bµ1,npµb2,np. 2. Nonparametric + parametric: µbpm,np =µb1,pmµb2,np. 3. Parametric + nonparametric: bµnp,pm =µb1,npµb2,pm. 4. Parametric + parametric: µbpm,pm=bµ1,pmµb2,pm.

For these estimators, equation (5.7) motivates the following mse estimators:

FIC(bµ_np,np) = 1

nmVb_1,npVb_2,np−µb²_1,npµb²_2,np, FIC(µb_pm,np) =



(bµ_1,pm−µb_1,np)²− 1

nVb_1,np+ 21

nVb_1,pm,np

 1

mVb_2,np+bµ²_1,pmbµ²_2,np

− 2µb_1,pmµb_1,npµb²_2,np, FIC(µb_np,pm) = 1

nVb_1,np



(µb_2,pm−µb_2,np)²− 1

mVb_2,np+ 21

mVb_2,pm,np



+µb²_1,npbµ²_2,pm

− 2µb²_1,npµb_2,pmµb_2,np, FIC(bµ_pm,pm) =



(bµ_1,pm−µb_1,np)²− 1

nVb_1,np+ 21

nVb_1,pm,np

 

(µb_2,pm−µb_2,np)²

− 1

mVb2,np+ 21

mVb2,pm,np



+µb²_1,pmµb²_2,pm− 2µb1,pmµb2,pmbµ1,npµb2,np. Also here, the FIC scheme chooses the µ estimators with the smallest FIC value.

5.4. FIC IN THE LOCAL MISSPECIFICATION FRAMEWORK 93 5.3.3 Generalizations

The formulae above with only two dierent samples may be generalized to three or more samples. The formulae and estimators then become much more complicated and are therefore omitted. There may also be situations where a type of comparison dierent from the two stated types, is of interest. In such situations the most natural approach is to write out the estimator in terms of quantities that can be estimated by one of the samples. Precise mse formulae may then be carried out by carefully rewriting each the squared bias and variance in terms of quantities that can be estimated from the data.