Estimation in Interacting Diffusions: Continuous and Discrete Sampling

(1)

Estimation in Interacting Diffusions: Continuous and

Discrete Sampling

Jaya Prakash Narayan Bishwal

Department of Mathematics and Statistics, University of North Carolina at Charlotte, Charlotte, USA

Received July 14, 2011; revised August 3, 2011; accepted August 10, 2011

Abstract

Consistency and asymptotic normality of the sieve estimator and an approximate maximum likelihood esti-mator of the drift coefficient of an interacting particles of diffusions are studied. For the sieve estiesti-mator, ob-servations are taken on a fixed time interval [0,T] and asymptotics are studied as the number of interacting particles increases with the dimension of the sieve. For the approximate maximum likelihood estimator, dis-crete observations are taken in a time interval [0,T] and asymptotics are studied as the number of interacting particles increases with the number of observation time points.

Keywords: Stochastic Differential Equations, Mean-Field Model, Large Interacting Systems, Diffusion Process, Discrete Observations, Approximate Maximum Likelihood Estimation, Sieve Estimation

1. Introduction

Finite dimensional parameter estimation in one-dimensional stochastic differential equations from continuous and discrete observations by maximum likelihood and Bayes methods are extensively studied in Bishwal [1]. Interacting particle systems of diffusions are important for modeling many complex phenomena, see Dawson [2] and Ligget [3]. Interacting particle systems are useful in constructing particle filter algorithms for finance and computation of credit portfolio losses, see Carmona et al.

[4]. Grenander [5] introduced the method of sieves for estimating infinite dimensional parameters. Sieve estimation for linear stochastic differential equations is studied in Nguyen and Pham [6]. Statistics for interacting particle models has not received much attention. Maxi- mum likelihood estimation in interacting particle system of stochastic differential equations was studied in Ka- songa [7]. In this paper we study nonparametric and parametric estimation in interacting particle system of stochastic differential equations by the method of sieves and the approximate maximum likelihood method respectively.

Consider the model of interacting particles of diffusions satisfying the Itô stochastic differential equations

n

 



 





 



 

=1

d = d

= 1, 2, ,

p

j l jl j j

l

,

X t t X t X t W t

j n

  





' where X t

 

=



X t X t₁

 

, ₂

 

, , X t_n

 



and

 



_j ;t0 , = 1, 2, ,



j  n are independent Wiener processes. Here

W t

 

2



 

_0, _{,d ,}



l L T t

   l= 1, , p are

unknown functions to be estimated based on observation of the process X in the time interval

 

0,T . Let

 

t =



   

t , t , ,

 

t



 ₁ ₂ p

 

and

 



₁ ₂ , _jp



'

= , ,

j x j x j x .

    x The processes

 

, = 1, 2, , j

X t j n are observed on

 

0,T .

The functions  _j, _j; j= 1, 2, , n are assumed to be known such that the system has a unique solution.

Here are some special cases: 1) Linear case:

 

   

 

=1

d _j = p _l d d _j , = 1, 2, ,

l

X t



 t X t t X t W t j n

2) Simple Mean-Field Model: Here the subsystems are interacting and exchangeable described by the system of SDE’s

     

 



 



 

d = d d d

0 = 0 , = 1, 2, ,

j j j n

j j

, j

X t t X t t t X t X t t W t

X x j n

   



where

 

1

   

 

=1

= n ,

n j j ,

(2)

 

t 0

  . The term containing can be viewed as an interaction between the subsystems that creates a tendency for the subsystems to relax toward the center of gravity of the ensemble. Here

 

t



 

t =



   

t ,



   t . The

case corresponds to sampling independent replications of the same process given below.

 

0



t



3) Independent case:

 

   

 

dX t_j = t X t t X t W t_j d  _j d _j , j= 1, 2,,n

p

We need the following assumption and results to prove the main results.

Assumptions (A1): Suppose that

are measurable and adapted processes satisfying

 

1

:= ; = 1, 2, , ; = 1, 2 , jl jl j

b  s  s j _ n l _

   

 

0 =1

1

d . .

n _t

jl jl lm

j

b s b s s c t a s as n

n



  

, = 1, 2 ,

l m p where c_lm

 

t are finite and continuous nonrandom functions of t

 

0,T . The limiting matrix I t

 

=



clm

 

t



_{l m}_{, =1,2, ,}__p is positive definite,

 

'_{I t}

  is increasing for all ___Rp_and_I

 

_{0 = 0}_. In the exchangeable case (A1) follows from McKean- Vlasov Law of Large Numbers. In particular, (A1) will be satisfied when _jl

 

X =_lX_j



and

 

=



j X Xj

  which corresponds to the independent replicated sampling on

 

0,T . We also need the following version of Rebolledo’s Central Limit Theorem for Martingales:

Let M n Zn,   be a sequence of local square integrable martingales with . Suppose the following condition holds:

 

0 = 0

n M

 



 





2



> 0

n n

s t

E M s I M s 



 





for all t

 

0,T , > 0;



and Mn

 

t c t

 

a.s.

for all t 0,T



, where c t

 

is a continuous increasing function with c

 

0 = 0. Then D

n M  M



, a continuous Gaussian martingale with zero mean and covariance function K s t

 

, =





, , 0,



=

c s t s t  T where

s s s

M M M 

 denotes the jump of M at the point s.

2. Sieve Estimator

Let P_ and P0 be the probability distributions of

 



X tt,  0,T



when 

 

 is the true parameter and

= 0

 respectively. Since _{  }

 

_L2



 

_0,_T _{,d ,}_t



_hence

P_ is absolutely continuous with respect to P0.

 



 





 



 



 





 



 





 

2 0 =1 =1 0 2 0

=1 =1 =1

d

= exp d

d 1 2 d .  _ _ _              _ 





p n _T

l jl j j

l j

p p n _T

l jl j

l m j

jm m

P

t X t X t X

P

t X t X t

X t t t

Our aim is to estimate the function 

 

 on

 

0,T

based on n interacting particles

 

1 2 , n

X  ,X  , X  of X t

 

on

 

0,T . The log- likelihood function is then

 



 





 



 



 



 







 



 

2 0 =1 =1 0 =1 =1 =1 2 = d 1 2 d .             





p n _T

n l jl j

l j

p p n _T

l jl

l m j

j jm m

L t X t X t X t

t X t X t X t t t

( )

j

Let Vn be increasing sequence of subspaces of

 





2 _0,

L T t



Vn

,d with finite dimension such that

1

n is dense in

dn

 





2 _0, _,d

L T t . The method of sieve (see Grenander [5]) consists of maximizing Ln

 

 on

n. Let k

V

 ,k= 1, 2, , be a sequence of independent

vectors of _L2



 

_0,_T ,dt



_{such that}

1, , _dn

  form a

basis of Vn for all . Then for

 

   k kn

 

d =1 n k , = n V

 



 , we have

 



 





 



 



 



 







 



 

          d 2 , 0 =1 =1 =1

d , 0 =1 =1 =1 =1 2 , =1 = d 1 2 d 1 = 2                                 _ _          _ _     

 



 





n

p n _T n

l k k jl j j

l j k

p p n _T n

l k k jl l m j k

dn

j jm m k k

k

'

n n n n n

L

t X t X t X t

t X t

X t X t t t

B A

where _ n ,_B n and

 

_A n _{are vectors and the matrix} with general elements

, = 1, 2, ,d

k k n



 

_{ }

_

_{ }

_

2

_

_{ }

_

_{ }

0

=1 =1

= d

p n _T n

k k jl j

l j

B



 t  X t  X t X tj ,

t = 1, 2, ,d _n

k  

 



 





 





 



 

, 2 0

=1 =1 =1

=

d , n

k r p p n _T

k jl j jm r

l m j

A

t X t X t X t t

    



, = 1, 2, ,d ._n

k r 

The restricted maximum likelihood estimator (sieve estimator) of  is

t

 

_{ }

d    

_{ }

=1

ˆn ₌ n ˆn k k k

 _  n _



where

 



     



1 2

ˆn ₌ ˆn_,ˆn_{, ,}_ ˆn dn

(3)

is the solution of

   n _ˆn ₌  n

A  B

Since _A n _{is invertible almost surely,}

 

  1  

ˆn ₌ _An _Bn_.

 

3. Properties of the Sieve Estimator

In this section we obtain consistency and asymptotic normality of the sieve estimator.

We focus on the interacting and exchangeable cases described by the system of SDE’s

 



 



 

d = ( ) d

d ,

0 = 0 , = 1, 2, ,

j j j n

j

j j

d

X t t X t t t X t X t t W t

X x j n

  



where

 

1

   

 

=1

= n ,

n j j ,

X t n



X t  t _ t _and

.

 

t 0

 

Here 

 

t =





   

t , t



.

0

Theorem 3.1 (Consistency) Under (A1), we have

 

_{   }





2

0 ˆ d

T _n _P

t t t

  



as and

such that

n  dn 

2

d 0

n n  .

Proof. The method of proof is similar to Nguyen and Pham [6] by using assumption (A1). We omit the details.

Theorem 3.2 (Asymptotic Normality) Let   =



1 , 2 , d 



be such that

n

n n n

n

   

 

_n 2 ₂

k

d =1

n

k   n 



as . Then under (A1), we

have

 



 

_{ }





_{ }



d 1

=1 ˆ 0,

n n D '

n

k k k

k

n



     N I T 

n 

as

and d_n  such that d3n 0

n  .

Proof. The method of proof is similar to Nguyen and Pham [6] by using Rebolledo’s CLT for martingales. We omit the details.

4. Approximate Maximum Likelihood

Estimator

In practice, one can not observe the diffusion process in continuous time. In this section we study parameter estimation based on observations at discrete time points. Let P_ and P₀ be the probability distributions of

 



X t_t,  0,T



when  is the true parameter and

= 0

 respectively. It is well known that P_ is absolutely continuous with respect to P₀.

The model is

 



 





 



 

=1

d = d d

= 1, 2, ,

p

j l jl j j

l

,

X t X t t X t W

j n

  





t

where

 

=



1

 

, 2

 

, ,

 



' n

X t X t X t X t and

 



_j ;t0 , = 1, 2, ,



j  n are independent Wiener processes. Here

W t



1 2



= , , , p

   

, , = 1,

jl j

is the unknown parameter. The functions   j , ; = 1, ,n l  p

are assumed to be known such that there exists a unique solution X t

 

to the above SDE. Approximate maximum likelihood estimation for the one dimensional case



n= 1



has been extensively studied, see Bishwal [1]. The approximate log-likelihood based on observations

 

, 1

 

2 , , 1

 

N , = 1, 2, ,

1 1

X t X t X t j n with

=

i

t iT N . It is known that AMLE is consistent as and

T   T N0 and satisfies LAN (local asymptotic normality) and asymptotically efficient as

and

T   2 3 ₀

N  n 

T N , see Bishwal [1]. In this paper we assume is fixed, and .



T

The Radon-Nikodym derivative (likelihood) is given by

 







 



 







 





 



2 0

=1 =1 0

2 0

=1 =1 =1

d

= exp d

d

1 _{d .}

2

p n _T

l jl j j

l j

p p n _T

l m jl j jm

l m j

P

X t X t X t P

X t X t X t

 _ _ _

    



  



 



 





t

Our aim is to estimate the parameter  based on particles X1

 

 ,X2

 

 , , Xn







of X t

 

on

 

0,T

n

.

The approximate log-likelihood based on observations

 

1 ,

 

2 , ,

 

, =1, 2,

j j j N ,

X t X t X t j n with t iT Ni=

is defined as

 







 





 



 







 



 









,

2

1 1

=1 =1 =1

2

1 1

=1 =1 =1 =1

1 1

=

1 2

.



  

   





 



 



 

 



N n

p n N

l jl i j i j i j i

l j i

p p n N

l m jl i j i

l m j i

jm i i i

L

X t X t X t X t X t X t

X t t t

1 

1

 Here we have used the approximation of the stochastic integral and the ordinary integral as in Bishwal [1].

Equating the derivative of the log-likelihood function to zero provides the estimating equations

 







 





 



 







 



 









2

1 1

=1 =1

( ) 2

, 1 1

=1 =1 =1

1 1

ˆ =

,

 

  





 



 



 



 

n N

jl i j i j i j i

j i

p n N n

N l jl i j i

l j i

jm i i i

X t X t X t X t

X t X t

X t t t

= 1, 2, ,

(4)

.

and the approximate maximum likelihood estimator (AMLE)

 



     



,1 ,2 ,

ˆn ₌ ˆn_,ˆn _{, ,}_ ˆn

N N N N p

   

5. Properties of the Approximate Maximum

Likelihood Estimator

In this section we obtain the consistency and asymptotic normality of the approximate maximum likelihood estimator.

Theorem 5.1 (Consistency) Under (A1), we have

 

ˆn P

N

   as N  and n .

Proof. The method of proof is similar to Kasonga [7] by using assumption (A1) with the aid of discrete approximations of the stochastic and ordinary integrals in Bishwal [1]. We omit the details.

Theorem 5.2 (Asymptotic Normality) Under (A1), we have

 



ˆn



D



_0, 1

 



N

n  _ _ N I T _asN _and

.

n 

Proof. The method of proof is similar to Kasonga [7] by using Rebolledo’s central limit theorem for martingales with the aid of discrete approximations of the stochastic and ordinary integrals in Bishwal [1]. We omit the details.

6. AML Estimation in Mean-Field Model

Let us consider approximate maximum likelihood estimator (AMLE) for the simple mean-field model

 



 



 

d = d d d

0 = 0 , = 1, 2, ,

j j j n j

j j

,

X t X t t X t X t t W t

X x j n

   

where

 

1

 

=1

= n , ,

n j j

X t n



X t  _ _and_{ }₀_{. The}

case  = 0 corresponds to sampling independent replications of Ornstein-Uhlenbeck processes on

 

0,T . Our parameter here is  =



 ,



   

ˆn n

.

The AMLE is N =



ˆN ,ˆN n



with

 

, ,

ˆn = n N n N ,

N

n N n N

S T R T

F T G T

 



 





 

, , , ,

, , ,

ˆn ₌ n N n N n N n N

N

n N n N n N

G T S T F T R T

F T G T G T

 



where

 

1

 



 

, 1

=1 =1

= n N

n N j i j i j i

j i

S T N X t X t X t

 



 







 



, 1

1 1

=1 =1

= ,

n N n N

j i n i j i j i

j i R T

N X t X t X t X t

    



1

 

₁



 



2





, 1

=1 =1

= n N ,

n N j i i i

j i

F T N



X t t t

 

₁



 



2





, 1 1

=1 =1

= n N .

n N j i n i i i

j i

G T N



X t X t t t1

Suppose =1

 

0

1

0 n

j j x

n



 almost surely and

 

2 2

0 =1

1

0 n

j j x

n

2 0

 

 



almost surely as . Then

the estimator

n 

 n P N

   as N  and n  and

 



ˆn





I 1

 

T



N

n  _ _DN 0,  _as __and_n_{ } where

 

= F T

 

_{ }

G T

_{ }





I T

G T G T

 

_ 

 



with

 

₌ 02



_e2 ₁



 

2

T _,

F T   G

   T

 









 









2 2 2

0 2

e 1

1

= .

2 2

4

T T

e T

G T

 

  

   

 



 _ 

 

 



In the classical case when  = 0, the AMLE is given by

 



 



 









1 1

=1 =1

2

1 1

=1 =1

ˆ = .

n N

j i j i j i j i

n

N n N

j i i i j i

X t X t X t X t t t

  

 





Sampling independent Ornstein-Uhlenbeck processes on

n

 

0,T and letting and

give weak consistency and asymptotic normality of the AMLE:

n  N 

 n P

ˆ_N

   and



 







2 2 0

2

ˆ 0,

e 1

n D

N _T

n   N _



 

 

 

  

 

as N  and n .



1 ,



Remark 1: One can look at this problem in a different way. If one observes the first Fourier modes in the expansion of the solution (the finite dimensional projection of the corresponding random field) of a parabolic stochastic partial differential equation (SPDE) and let the dimension of the projection increase

n

(5)

7. References

while remains fixed, the Fourier modes are independent Ornstein-Uhlenbeck processes, see Bishwal [1]. Another important point to be noted here is the connec- tion between the method of sieves and the spectral Fourier asymptotics in SPDE.

T

[1] J. P. N. Bishwal, “Parameter Estimation in Stochastic Differential Equations,” Lecture Notes in Mathematics, Vol. 1923, Springer-Verlag, Berlin Hiedelberg, 2008. [2] D. Dawson, “Critical Dynamics and Fluctuations for a

Mean-Field Model of Cooperative Behavior,” Journal of Statistical Physics, Vol. 31, No. 1, 1983, pp. 29-85. doi:10.1007/BF01010922

Remark 2: Testing of hypothesis of noninteraction versus interaction of the subsystems, i.e., H0: = 0

versus H1: 0 based on discrete observations of the

system can be studied. Since _n



ˆ n



N

  has approximately _N



_0,_J1

 

_T distribution where

 



[3] T. Ligget, “Interacting Particle Systems,” Springer-Ver- lag, New York, 1985.

 





 

1 F T _[4] _{R. Carmona, J. P. Fouque and D. Vestal, “Interacting}

Particle Systems for the Computation of Rare Credit Portfolio Losses,” Finance and Stochastics, Vol. 13, No. 4, 2009, pp. 613-633. doi:10.1007/s00780-009-0098-8

=

J T

F T G T G T



 , hence under H0,

 

, ˆ

n

has approximately N

 

0,1

n N T N

nJ 

[5] U. Grenander, “Abstract Inference,” Wiley, New York, 1981.

distribution where

 



,

 

,

_{ }

 



,

 

,

= n N n N n N

n N

F T G T G T

J T

F T



is an estimate [6] H. T. Nguyen and T. D. Pham, “Identification of Nonsta-tionary Diffusion Model by the Method of Sieves,” SIAM Journal of Control and Optimization, Vol. 20, No. 5, 1982, pp. 603-611. doi:10.1137/0320045

 

J T

 

of based on discrete observations with

 

P

,

n N

J T  J T as and . Thus the

null hypothesis is rejected if

N  n 

 

, ˆ >

n

2

N z

n N nJ T

[7] R. A. Kasonga, “Maximum Likelihood Theory for Large Interacting Systems,” SIAM Journal on Applied Mathe- matics, Vol. 50, No. 3, 1990, pp. 865-875.

doi:10.1137/0150050 



where  is the chosen size of the test and is normal quantile.