for Simultaneous Estimation of Parameters of
Several Probability Distributions
Rahul Gupta
Department of Statistics, University of Jammu Jammu (J&K)-180006, India
[email protected] Yodh Raj
Department of Statistics, University of Jammu Jammu (J&K)-180006, India
[email protected] Sunil Kumar
Department of Statistics, University of Jammu Jammu (J&K)-180006, India
Abstract
The fixed sample size procedure sometimes fail to deal with the estimation problem in which it is desired to control the combined risk associated with the estimation of parameters of various probability distributions simultaneously. In this paper for a general probabilistic model is proposed an “accelerated” sequential class to minimize the combined risk for simultaneous estimation of parameters of several probability distributions. The proposed class is shown to provide solutions for many estimation problems under different probabilistic setups for “given precision” problems. Besides, positive and negative moments for the stopping times are obtained and they are used for deriving the asymptotic expression for the “regret” associated with the class of “accelerated” sequential estimation procedure.
Keywords: Loss function, Accelerated sequential procedure, Asymptotic distribution,
Regret.
1.1 Introduction
Sometimes, it is desirable to control the combined risk associated with the estimation of parameters of various populations and such problems give rise to the simultaneous estimation problems. The fixed sample size procedures for such problems fail and affirmative solutions are desired. In this direction, Raatikainer(1987) proposed a sequential procedure for simultaneous estimation of percentiles under a general set-up. Then Mukhopadhyay, Hamdy and Darmanto(1988) considered the negative exponential populations and desired the simultaneous confidence interval estimation of the parameters as a follow up of certain selection and ranking problems. Later Mukhopdhyay(1992) provided multi-stage procedures for the simultaneous point estimation of the parameters of several negative exponential populations. For some further work on related simultaneous estimation procedures one can cite the papers of
Raatikainer(1993), Mukhopadhyay and Solanky(1998), Aoshima and
In this paper we have considered a generalized problem of simultaneous estimation of parameters of several populations
f
x
i,
i,
i
, i = 1,2,……,k; where
i and
i are the unknown parameters, under a family of general loss function and a linear cost function. The problem of fixed sample size procedure to deal with such problem is established in Section 1.2. A class of “accelerated” sequential procedure to tackle the problem is proposed in Section 1.3 and positive and negative moments for the stopping times are obtained and they are used for deriving the asymptotic expression for the “regret” associated with the class of “accelerated” sequential estimation procedure. Finally in Section 1.4.illustrations of estimation problems are provided which can be dealt with the help of the proposed class.1.2 The Set-Up of the Problem
Let
Xij , j1,2,.... be a sequence of independent and identically distributed (iid) random variables from the ith,i ,12,3,....,k t-variate (t1) population
i i i;θ ,ψ X f , where θ andi i
ψ are the unknown parameters. Denoting byRtandR, respectively, the t- dimensional
euclidean space and the positive –half of the real line, let t i R
θ andψ R
i . Having
observed a random sample Xi1,...,Xinj of size nj
t1
from the i population letth
i
j i i1 in in θˆ X ,...,X θˆ and
i
i i i1 in in ψˆ X ,...,Xψˆ be the estimators of θ andi ψ ,i
respectively, and satisfying the following assumptions:
(A ): There exist a known txt positive definite matrix1 Q , a number δi
0
,
1
and aninteger r(≥1) such that
2 r i in i i in 1 i i ˆ ˆ Q ˆ ˆ ~ n i i ,where
χ
2r denotes a chi-squarer.v’s with r degrees of freedom.(A ): For all2 ni mt ,1 θˆ andini ψˆiniare stochastically independent.
(A ): There exist integers q(≥1) and s(≥1) such that, for all3 ni s1,
n s 1 j q j i in i i i /ψz
ψ s n q withz
qχ
2q j ~ .Our goal is to estimate the vector
1,...,k
pointwise. For
k 1 i i n n , an obvious estimator ofθ is
k 1 kn n 1 n θˆ ,...,θˆθˆ . Let the loss incurred in estimating θ by θˆnbe
k
1 k
1 i u i in i i in n A θˆ θˆ Q θˆ θˆ C n ... n θˆ , θ L
i i , (1.1)where A (>0) is the known weight and C (>0) is the known cost per unit sample observations from each population.
k i i k i u i u i n c K u r C n Rn
1 1 , ,
, where (1.2)
u,r,
2
A
r2 u
r2 K u . The value * in of n from thei i population, which minimizes the risk (1.2), is given byth
1 1 1 * , , u u i u i C r u ukn
(1.3)And substituting n =i n in (1.2), for*i
k 1 i * i * nn , the corresponding minimum risk is
1 1
** C C u n
Rn (1.4)
However, in the absence of any knowledge aboutψ ’s, no fixed sample size procedurei
uniformly minimizes the risk simultaneously for all values of ψ ’s. In such a situation,i
motivated by (1.3), we propose the following class of ‘accelerated’ sequential procedure, determining the sample size as a random variable.
1.3. The Class C of ‘Accelerated’ Sequential Procedure
Take
m
max{
s
1
t,
1
}
to be the initial sample size from each population, where, as in Hall(1983), m is chosen so as to satisfy mo(C1(u1)) as C 0and1
)
n
m
sup(
lim
i 0c
. Let (0,1) be specified. Start sampling sequentially from the thi
population, with the stopping time
M
i defined byi
M
= ( 1) ( 1) 1 ˆ ) , , ( : inf u u u i i C ini r u uk n m n (1.5)Based on these
M
iobservations, we compute
ˆ
i Mi . Then we jump ahead and collecti i
M
N
more observations from thei
th population, where iM
= ( , , ) ˆ 1 , max ( 1) ( 1) 1 u u u i C ini r u uk M (1.6)After stopping, estimate by ˆ(N) where N =
k 1 i
N
i and ) ( ˆ N =
ˆ(1N1)...ˆk(Nk)
.It can be seen that the risk associated with the class C of ‘accelerated’ sequential estimation procedures (1.5)-(1.6) is
u i u k i i N c cu n E N R ( 1) 1
E N c u n c k i i ) 1 ( 1 (1.7) withN
i determined by present rule.The “Regret” associated with the estimator ˆ(N) of
is
c R
c R
cRg N n
(1.8) Now we state and prove some lemmas.
Lemma 1: For the class Cof accelerated sequential procedures and all
,t
s
2
u
/(
u
1
)
max
m
, as C0,
1
u
/(
u
1
)
1
/
2
o
(
1
)
)
1
u
(
q
u
s
n
N
E
i *i
(1.9) and ) ( ) 1 ( 2 ) ( * 1/( 1) 2 2 u i i q uu n o C N Var (1.10) Proof : Denoting by ) 1 /( ) 1 /( 1 ) 1 /( ) 1 /( 1 ) , , ( ) , , ( 1 u u M u u u M u Mi i C i r u K u C r u K u U We can write)
N
(
E
i = I + II, (1.11) where I= 1 ) , , ( /( 1) ) 1 /( 1 u u M u i i C i r u K u M I N E (1.12) II= ( , , )
/( 1)
( ) ) 1 /( 1 i i M u u M u U E E C r u K u (1.13) It follows from Hall (1983) that, as C 0, I= o(1) andU
Miis uniform over (0,1). Thus, we conclude from (1.11), (1.12) and (1.13) that, as C 0,
N
iE
=
/( 1)
) 1 /( 1 ) , , ( u u M u i E C r u K u + 1/2 + o (1) (1.14)Now we evaluate
Mu/(u 1)
iE
. To this end, we rewrite the stopping rule (1.5) as i M =
1 (u 1)/u i u / ) 1 u ( 1 i 1 / ) u ( * i n 1 j ) q ( j 1 i m s: q Z n n (1 sn ) n inf i (1.15)Comparing (1.14) with equation (1.1) ofWoodroofe (1977), we obtain in his notations,
i i n 1 j ) q ( j 1 nq
Z
S
,
1
(
u
1
)
/
u
,
u
/(
u
1
)
,
n
*i, u u i i sn n L( )(1 1)( 1)/ ,L
s
(
u
1
)
/
u
,
1
o
, and
2
2
q
1. It follows from hisTheorem 2.4 that, for all
m
s
2
u
/
q
(
u
1
)
, as C0,),
1
(
o
))
1
u
/(
u
1
(
)
1
u
(
q
u
s
)
1
u
(
u
n
)
M
(
E
i *i
(1.16) where
is specified.Let us consider the difference
/( 1) ) 1 /( 1 ) , , ( u u M i u i c C i r u K u M D (1.17)
It follows from Woodroofe (1977) that the mean of the asymptotic distribution of
c
D
is) 1 (uu .
Thus we obtain from (1.15) and (1.16) that, for all
2 / ( 1)
s u q u m , as C0,
/( 1)
) 1 /( 1 ) , , ( u u M i u i E C r u K u =
1 /( 1)
(1). ) 1 ( 1 * u u o u q u s ni (1.18)Result (1.7) now follows on substituting the value of (1.7) in (1.14). Let
* 1/2 */)
(Mi Mi ni ni
h . It follows from a result of Bhattacharya and Mallik (1973)
that h(M )L N
0,2u2/q(u1)2
i , as c0. Moreover, from Theorem1 of
Woodroofe(1977), 2( ) i
M
h is uniformly integrable for all m
s2u/q(u1)
. Hence, for all m
s2u/q(u1)
, as C → 0,
2( )
i M h E = (1) ) 1 ( 2 2 2 o u q u .And by the definition, ) (Ni
Var = 2 Var(M ).Therefore, i (1) ) 1 ( 2 ) ( 2 * 2 2 o u q u n N
Var i i , and (1.9) holds.
Lemma 2: As C0,
qm s u
ii n oC
N
p * ( )/2 .
Proof: It is easily proved along the lines of Lemma 3 of Chaturvedi, Pandey and Gupta
(1991).
Lemma 3: Let the r.v’s
W
is'
, i=1,2,…,k, be defined by 1
/ *
1 i i i N n W . Then, for 2 , 0 Wi is uniformly integrable for all m
s2u(2)/q(u1)
.Proof: On the event * i i n
N , 2 1 1
i i n and W
W , so that, on this event, positive,
as well as negative, powers of W are uniformly integrable. Furthermore,on the eventi *
i i n
N Wi 2 and Wi1 ni*/m.
Now, for2, applying Lemma 2, we get from some B (>0),
2 *
*2
*
i i i i i i I N n Bn p N n W E = o(cq(ms)/2u(2)/(u1)) = o (1), for allms2u(s2u)(2)/q(u1). Finally, for 2, once again applying Lemma 2, we get
2 *
*
i i i i i I N n Bp N n W E = o(1), for all ms1. And the lemma follows.
Lemma 4: For any 0 and all m
s2u/q(u1)
,
s2u
2
/q(u1)
, as 0 C ,
i N E =
2
1
( 1)
1
/2
1 2 2 * * 1 n q u u qs u u u q n n i i ) ( ( 1)/( 1) o c u (1.19) And
i N E =
1
2
1
( 1)
1
/2
2 2 * * 1 n q u u qs u u u q n n i i ) ( ( 1)/( 1) o c u (1.20)Proof : Using the expression
i N E = i n
2 i 2 i 2 i i * i i i n w n N E 2 1 n N E n 1Lemmas 1 and 3 and the fact that a.s 1 i
W as C0, we obtain for all
2 / ( 1), 2 2 / ( 1)
max s u q u s u q u m , as C0,
i N E =
) 1 ( 2 / 1 ) 1 /( 1 1 1 * * 1 o u u u q u s n ni i
) ( 1 2 2 1 * 1/( 1) 2 2 * 2 u i i n o c u q u n . And (1.19) follows.Furthermore, using Taylor’s expression for
i
N , Lemma 1 and 3, and the fact that 1
.
as i
W as C 0, we obtain for all
2 / ( 1), 2 2 / ( 1)
max s u q u s u q u m , as C 0,
i N E =
) 1 ( 2 / 1 ) 1 /( 1 1 1 * * ( 1) o u u u q u s n ni i
) ( 1 2 2 1 * 1/( 1) 2 2 *( 2) u i i quu n o c n , and (1.20) holds.Now we prove the main theorem of this section, which provides asymptotic expression for the ‘regret’ corresponding to the class C of ‘accelerated’ sequential estimation procedures.
Theorem: For the class C of ‘accelerated’ sequential procedures and for all
, 2 / ( 1) , 2 2 / ( 1)
max t s u q u s u q u m , as C0,
( ) 1 2 C o u q Cku c Rg .Proof: Substituting the values of Rn
C andRN
C from (1.4) and (1.7) in (1.8)respectively, we obtain
C Rg ( / ) ( ) ( ) (1/ 1) *. 1 1 *( 1) n u C N E C N E n u C k i i u i k i i u
(1.21)Now, applying Lemma 4, we obtain for any
0
and all
, 2 / ( 1) , 2 2 / ( 1)
max t s u q u s u q u m , as C0, we get
k i i i i g n u u u qs u u q u oC u q u n n u C c R u u u 1 2 * 2 * * 2 1 )) 1 ( )( 1 ( ) 2 ( 1 / 1 1
* 1 2 1 * 1 * 1 /( 1) 1/2 (1) 1/ 1 u u o C u nn u q u s n n C k i i i
,and the theorem follows .
1.4 Estimation Problems Having Solutions Provided by Class C of“Accelerated” Sequential Procedure
1.4.1 Simultaneous estimation of the means of several normal populations
Let us consider a sequence
X
ij , j= 1,2,…. of iidr.v’s from thei
th,i=1,2,……k univariate normal population ( , 2), i i i x
N , where
i
(
,
)
and
2
(
,
)
i are
the unknown mean and variance, respectively. Having recorded a random sample
i
in
i
X
X ,...,
1 of sizen
i(
2
)
from thei
thpopulation, let
i i n i n j ij iX
n
X
1 1 ) ( and 2 1 1 2 ) ((
1
)
(
( ))
i ini i n j ij i n in
X
X
s
.It can be verified (see Starr(1966)] that)
(
)
(
A
1
A
3 are satisfied for
i
i,
i
i2,
ˆ
ini =X
i( in), 2
ˆ
i n i
=s
i2 ni , r=q=s=t=1, Qi= I1x1= 1 and δ =1.1.4.2. Simultaneous estimation of the mean vectors of several multinormal populations
Let us consider a sequence
X
ij , j= 1,2,…. of iidr.v’s from thei
th,i=1,2,……kunivariate normal population ( , 2 )
, i i i i
p x
N , where
i is the px1 unknown meanvector, 2(0,) i
are the unknown scalar and
i is a known pxp positive definite matrix. Given a random sampleX ,...,
i1X
iniof size ni( p1) from thei
thpopulation, let
i i n i n j ij iX
n
X
1 1 ) ( and
2 1 1 2 ) ((
1
)
(
)
ˆ
( )
i i n i i n j ij i n ip
n
X
X
.Itcan be verified (see, Wang, 1980) that
(
A
1)
(
A
3)
hold for
i
i,
i
i2,
ˆ
ini = )( in i
1.4.3. Simultaneous estimation of the regression parameters of several linear models
Consider the
i
th linear model Yi(ni)=Xi(ni) ii(ni), i=1,2,…..k.,whereYi(ni) is an observed ni*1random vector, X i(ni) is a
n
i*
p
Matrix of rank p, i is the p*1 vector of unknown regression parameters, and i(ni)is the disturbance term following (0, 2 )
i i i n
n I
N distribution. The ordinary least squares
estimator of i is ˆ ( ) i n i =
(
X
i(ni)X
i(ni))
1X
i(ni)Y
i(ni) and we use
1 2 ) (ˆ
n
p
i n i i
Y
i(ni)[
I
ni
X
i(ni)(
X
i(ni)X
i(ni))
1X
i(ni)]
Y
i(ni) to estimate 2 i
. It can be seen [see, Judge and Bock (1978, P.20)] that(
A
1)
(
A
3)
are satisfied for r = t = p,i= i,
i
i2,
ˆ
ini = ˆ i(ni), 2ˆ
i n i
=
ˆ
i2 ni , q=1,s=p,Q
i
n
i1(
X
i(ni)X
i(ni))
1 and
1
.1.4.4. Simultaneous estimation of the location parameters of several negative exponential populations
Let
X
ij , j=1,2,….. be a sequence of the iidr.v’s from thei
thnegative exponential populations, ) , ; (xi i i f = i i i i x 1exp[( ) ]; i ix
, i=1,2,…..,k, where i(,) and ) , 0 ( i are the unknown location and scale parameters, respectively. For a random sample
X ,...,
i1X
ini of size ni(2) from thei
thpopulation, we define) ,..., min( 1 ) (ni i i Xi Xin X and
i i n i i n j ij i n i n X X 1 1 ) ( 1 ( ). ˆ ( ) Assumptions
(
A
1)
(
A
3)
are satisfied [see Basu (1971)] for r=2, t=1,
i =
i,
i
i,
i in
ˆ
= ) ( in iX
,
ˆ
i ni =
i2 ni , q=2, s=1,Q
i
I
11
1
and
1
2
.1.4.5. Simultaneous estimation of the parameters of several Pareto distribution
Let
X
ij , j=1,2,….. be a sequence of the iidr.v’s from thei
th(i= 1,2,…..,k) first kind of Pareto distribution)
,
;
(
x
i i if
= 1 1 i 1 i1 i i i
x
;x
i
i
0
,
i
0
, where
i and
i arerespectively , unknown scale and shape parameters. Based on random sample
i
in
i
X
X ,...,
1 of size, forn
i(
2
)
X
i(ni)
min(
X
i1,...,
X
ini)
, we use ) ( in i
= ) (log
i n iX
and
i i n i i n j ij i n i n X X 1 1 ) ( ) ( log 1 ˆ as the estimators of log
i and
i are respectively.We observe that [see, Wang (1973)](
A
1)
(
A
3)
are satisfied for r = 2, t = 1,i= log
i,
i
i,
ˆ
ini = ) ( in i
, ni i
ˆ
=
ˆ
i ni , q=2, s=1,Q
i
I
11
1
and
1
2
.1.4.6. Simultaneous estimation of the means of the several inverse Gaussian populations
Let
X
ij , j=1,2,….. be a sequence of the iidr.v’s from an inverse Gaussian distribution) , ; (xi i i f =
i i i ix
x
2 2 i 2 1 3 iexp
2
x
2
; ix
>0, where
i
,
and 1(0,) i are the unknown mean and scalar parameters respectively. Given a random sample
X ,...,
i1X
ini of size,n
i(
2
)
from thei
thpopulation, we use , 1 1 ) (
i i n i n j ij i X n X
i i n i ij i n j i n in
X
X
1 1 1 1 ) (1
(
)
ˆ
) (
as the estimators of
i and are
i1respectively.We observe that (see, Chaturvedi,1985))
(
)
(
A
1
A
3 are satisfied for r = t = 1,i=
i,
i
i1,
ˆ
ini = ) ( in i
X
, ni i
ˆ
=
ˆ
i(ni), q = s =
=1 andQ
i(ni)
I
11
1
.1.4.7.Simultaneous estimation problem related to multiple comparison procedures
Let us consider the linear regression models of sections 3. In multiple comparison procedures [see Hochberg and Tamhane(1987)] for a brief discussion], one may be interested in estimating the parametric functions of the components of a lx1 (
1
l
p
) subvector i of i.Let ) ( ˆ i n i be the corresponding subvector of ˆ ( )
i
n i
and
V
i is 1x1submatrix of(
X
i(ni)X
i(ni))
1 corresponding to the ) ( ˆ i n i part of ( ) i n i . For a known
p x1 matrix
G
i=(
g
i1,...,
g
ip
)
,
i= ( 1 ,..., ) i ip i i g g =G
i i . We use the
least-squares estimator
i=G
i ) ( ˆ i n i to estimated by si2(ni) i1 Yi Yˆ(ni) 2 , whereˆ
( ) i n iY
= ) ( ) (ni ˆini iX ,
n
i
s
io is the error degree of freedom ands
io = rank(
X
i( in))
.Assumptions(
A
1)
(
A
3)
are satisfied for r = t = p,i= i,
i
i2,
ˆ
ini =Y
ˆ
i(ni), ni i
ˆ
=s
i2(ni) , q = 1, s =s
io,
=1 andQ
i=(
G
iV
iG
i
)
1.1.4.8. Simultaneous estimation of the means of several random one-way models
Let us consider the
i
th(i=1,2,……,k) random one way model:ijl
Y
=
i +
ij+
ijl, j=1,2,…...,n
i; l= 1,……..,r
i, where
ij’s are independently distributed as ( 2), il
N ,
ijl’s are independently distributed asN
(
, ie
2)
,
ijand
ijlare independent for all i ,j and l and ij (0,)and)
,
0
(
ie
are unknown. We assume thatr
i-the number of samples per treatment isknown. Let
Y
i =
ni i j r l ijl i ir
Y
n
1 1 1)
(
be the over all sample mean from thei
th populationand MST and MSE denotes, respectively, the usual ANOVA mean squares for the treatments and errors. We notice that the assumptions
(
A
1)
(
A
3)
are valid for r = t = 1,i = i,
i
(
r
i
il2
ie2)
,
ˆ
ini =Yˆ
i, ni i
ˆ
=MST =
i n j i i ij i nY Y r 1 2 . ) 1 ( ) ( , q =n
i, s =
=1 andQ
i= ri I1x1ri. References1. Basu, A.P. (1971). On a sequential rule for estimating the location parameter of an exponential distribution. Nav.Res.Logist.Quart.,18,329-337.
2. Bhattacharya, P.K. and Mallik, A. (1973).Asymptotic normality of the stopping times of some sequential procedures. Ann. Statist., 1, 1203-1211.
3. Chaturvedi, A. (1985). Sequential estimation of an inverse Gaussian distribution with prescribed proportional closeness. Cal. Statist. Assoc. Bull., 35, 215-219. 4. Chaturvedi, A. Pandey, S.K. and Gupta, M.(1991).On a class of asymptotically
risk-efficient sequential procedures. Scand, Actur. Jour., 1:87-96.
5. Ghosh, D. (2005). Simultaneous estimation procedures and multiple testing: a decision- theoretic framework. Working Paper Series- Working Paper 54. The
6. Efron, B. (2004). Selection and estimation for large scale simultaneous inference. estimation of the mean. Jour. Amer. Statist. Asso.,96, 96-104.
7. Hall, P. (1983). Sequential estimation saving sampling operations.
Jour.Roy.Statist.Soc. B 45, 303-310.
8. Hochberg, Y. and Tamhane, A.C. (1987). Multiple comparison procedures. John
Wiley and Sons Inc., U.S.A.
9. Judge, George G. and Bock, M.E. (1978). The Statistical Economics.
North-HillPublishing Company, New York, U.S.A.
10. Raatikainer, K.E.E. (1987). Sequential procedure for simultaneous estimation of several percentiles. Simulation, Vol. 49, 4,159-164.
11. Raatikainer, K.E.E. (1993). A sequential procedure for simultaneous estimation of several means. ACM Trans. On Modelling and Comp. Simulation., 3,2,108-133. 12. Aoshima, M. and Mukhopdhyay, N. (1998). Fixed width simultaneous confidence
intervals for multinormal means in several intraclass correlation models.
Jour.Multiv.Anal.,66,1, 46-63.
13. Mukhopdhyay, N. (1992). Simultaneous point estimation problems for two parameter negative exponential population. Jour.Ind.Statist.Assoc.,30, 33-41. 14. Mukhopdhyay.N, Hamdy, H.I and Darmanto,S. (1988).Simultaneous estimation
after selection ranking and other procedures: the negative exponential case.
Metrika, 35, 275-286.
15. Mukhopdhyay, N. and Solanky, T.K.S. (1998). Multi-stage methodologies for fixed with simultaneous confidence intervals for all pairwise comparisons. Jour.
Stat. Plan. Infer.,73, 1-2, 163-176.
16. Starr, N. (1966). The performance of a sequential procedure for the fixed width interval estimation of the mean. Ann.Math.Statist.,37, 36-50.
17. Wang, Y.H. (1973).Sequential estimation of the scale parameters of the Paretodistribution. Commun. Statist.-Theo. Math., A2, 145-154.
18. Wang, Y.H. (1980). Sequential estimation of the mean of a multinormal population. Jour. Amer. Statist. Assoc., 75, 977-983.
19. Woodroofe, M. (1977). Second order approximations for sequential point and interval Ann.Statist.,5, 984-995.