Wayne State University Dissertations
1-1-2016
Ergodicity Of Stochastic Switching Diffusions And
Stochastic Delay Systems
Hongwei Mei
Wayne State University,
Follow this and additional works at:http://digitalcommons.wayne.edu/oa_dissertations Part of theMathematics Commons
This Open Access Dissertation is brought to you for free and open access by DigitalCommons@WayneState. It has been accepted for inclusion in Wayne State University Dissertations by an authorized administrator of DigitalCommons@WayneState.
Recommended Citation
Mei, Hongwei, "Ergodicity Of Stochastic Switching Diffusions And Stochastic Delay Systems" (2016).Wayne State University Dissertations.Paper 1564.
AND STOCHASTIC DELAY SYSTEMS
by
HONGWEI MEI
DISSERTATION
Submitted to the Graduate School of Wayne State University,
Detroit, Michigan
in partial fulfillment of the requirements for the degree of
DOCTOR OF PHILOSOPHY 2016 MAJOR: MATHEMATICS Approved By: ———————————————————– Advisor Date ———————————————————– ———————————————————– ———————————————————– ———————————————————–
To my families.
It is my great pleasure to express my gratitude to those who have helped me in this dissertation. Above of all, I would like to thank my advisor Professor George Yin for his guid-ance, encouragement and patience over the past five years. Without his carful and persistent help, the dissertation would not have been possible.
I also would like to thank my committee members, Professor Fatih Celiker, Professor Leyi Wang and Professor Peiyong Wang, for their time and discussion on my research. My special thanks are expressed to Professor Jianhai Bao, Professor Xiaoyue Li and Professor Fuke Wu for their kind suggestions on research as well as life.
During my years of graduate study at Wayne State University, I worked on a number of research projects at different times. These projects were partially supported by the National Science Foundation. The supports are greatly appreciated.
I wish to thank my friends with whom I share these enjoyable years here. Wish you best of luck in your lives. Lastly, I would like to share this moment with my family. I am indebted to my parents, my sister and Rui for their endless care, love and support.
Dedication . . . ii
Acknowledgements . . . iii
List of Figures . . . vi
Chapter 1: Introduction . . . 1
Chapter 2: Numerical Algorithms for Stochastic Switching Diffusions . . . 3
2.1 Introduction . . . 3
2.2 Formulation and Preliminary Results . . . 5
2.3 Approximation of Ergodic Means . . . 10
2.4 Convergence Rates for Approximation to Ergodic Means . . . 24
2.5 Examples and Remarks . . . 32
Chapter 3: Stochastic Delay Systems with Infinite Delay . . . 37
3.1 Introduction . . . 37
3.2 Formulation . . . 40
3.3 Solution Mapxt and Properties . . . 54
3.4 Mean-square Bounds and Large-time Estimates for xt . . . 60
3.5 Invariant Measure . . . 67
3.6 Properties of SpaceCr . . . 70
Chapter 4: Stochastic Integro-differential Equations . . . 72
4.1 Introduction . . . 72 iv
4.3 Regularity . . . 80 4.4 Ergodicity . . . 90 4.5 Proof of Theorem 4.5 . . . 101 References . . . 105 Abstract . . . 112 Autobiographical Statement. . . 114 v
Figure 1 : Cn as a function of Γn . . . 34
Figure 2 : Cn as a function of Γn . . . 35
CHAPTER 1 INTRODUCTION
This dissertation is devoted to the study of ergodicty of stochastic systems. It consists of two main parts. The first part is concerned with ergodicity of numerical solutions for stochastic switching diffusions. The second part focuses on ergodicity of stochastic delay systems with infinite delay.
Chapter 2 focuses on numerical algorithms for approximating the ergodic means for suit-able functions of solutions to stochastic differential equations with Markov regime switching. Different from the usual diffusion, a switching diffusion is one in which continuous states are coupled with a discrete events. In our case, the discrete event is given by a finite state Markov chain. Taking into account the discrete component, switching diffusions are more suitable and more realistic for many applications that include the interactions of continu-ous and discrete motions. Although the systems might be seemingly similar to the diffusion counterparts, their behavior can be quite different. An important topic that attracts a lot of attention is when such processes are stable in some sense. Therefore in-depth study of such properties as recurrence, positive recurrence, ergodicity, stability, and invariance principles was carried out in the past decades. Nevertheless, for vast majority of switching diffusions, the processes cannot be solved in a closed form since they are highly nonlinear and complex. As a result, not only are numerical methods important for mathematical development, but also they are crucial for practical purposes. Along this line, there are a lot of works being devoted to comparing the invariant measures of switching diffusions and their numerical algorithms provided that the process is ergodic. Although numerical algorithms have been studied extensively, the work on the convergence rate to the ergodic measure of a numerical
algorithm is still scarce. This chapter is devoted to this topic. By proposing a recursion algo-rithm with decreasing steps, we obtain the convergence and rates of the numerical algoalgo-rithm to the ergodic measure.
Chapter 3 and Chapter 4 focus on the ergodicity for a class of stochastic delay equa-tions with infinite delay. The motivation of a stochastic delay equation stems from non-instantaneous transmission phenomena, for example, high velocity fields in wind funnel ex-periments, or other memory processes, or specially biological motivations such as species growth or incubating time on disease models among many others. An essential difference of stochastic process with delay from a process without a delay is that the solution process is no longer Markov. To overcome this difficulty, a solution mapping process is introduced and proved to be a Markov process in some appropriate phase space. Different from the existed works on stochastic delay equations with finite delay, Chapter 3 is devoted to investigat-ing the asymptotic results of stochastic delay equations with infinite delay. By proposinvestigat-ing a suitable phase space, we prove the solution mapping process is Markov and ergodic with a unique invariant measure under some suitable conditions. While in Chapter 4, for stochastic integro-differential equations, we use a totally different approach to markovianize the solu-tion process instead of using solusolu-tion mapping process. We map the solusolu-tion process into a new Polish space and prove the resulting process is Markov and ergodic in the new Polish space under some suitable conditions. Moreover, a weak sense Fokker-Planck equation is derived for the underlying process.
CHAPTER 2 NUMERICAL ALGORITHMS FOR
S-TOCHASTIC SWITCHING
DIFFUSION-S
2.1
Introduction
Recently, resurgent effort has been devoted to studying switching diffusion processes; see [37, 64] among others. Having two components (a continuous component X(t) and a discrete
component α(t)), such processes are featured in the coexistence of continuous dynamics of
the usual diffusion processes as well as discrete events formulated by a switching process. The main thrust on understanding such processes stems from the needs of more realistic models with interactions of continuous and discrete motions. Although the systems might be seemingly similar to the diffusion counterparts, their behavior can be quite different. In [64], a systematic treatment of switching diffusion processes was provided; an in-depth study of such properties as recurrence, positive recurrence, ergodicity, stability, and invariance principles was carried out. Necessary and sufficient conditions for positive recurrence were obtained using solutions of systems of coupled partial differential equations. The expected value of the recurrence time was shown to be the smallest positive solution of a Dirichlet problem. In addition, it was proved that if a switching diffusion is positive recurrent, it has a unique invariant measure. As a consequence, a law of large numbers for a long-run average of a suitable function was obtained for positive recurrent processes. Such results are important for subsequent study of long-run average type stochastic controls or other type of stochastic controls in an infinite horizon [26]. Furthermore, in our recent work [43], laws of iterated logarithms were established for switching diffusion in a finite horizon. These results paved a way for studying many important properties. Nevertheless, for vast majority of switching
diffusions, the processes are highly nonlinear and complex. Therefore, more often than not, they cannot be solved in closed form. As a result, not only are numerical methods important for mathematical development, but also they are crucial for practical purposes.
In the literature, numerical approximations to diffusions have received much needed at-tention; see for example, the well-known references [34] and [44] among others. Moreover, the studies on convergence to invariant measures [7] and convergence of numerical approximation with reflections [52] were carried out. In [56], a second-order scheme for approximating diffu-sion processes was proposed and the error analysis of the convergence of a sample average of a suitable function for the numerical solution to the ergodic mean (the integral of the function w.r.t. the stationary measure) was analyzed in details. Treating switching diffusions, recent work on numerical solutions can be found in [63]. Effort has also be devoted to approxi-mating the invariant measures of the processes when they exist. Stability in distribution was considered in [65], whereas [38] focused on designing numerical algorithms for approximating the invariant distributions. However, convergence rates for numerical approximation to the invariant measures or convergence rates for averages with respect to invariant measures have not been developed for switching diffusions to the best of our knowledge. In this paper, we consider numerical algorithms with a sequence of decreasing step-sizes to approximate the switching diffusions. We focus on the convergence rates of a sample average of a suitable function to the average with respect to the invariant measure. First a law of large numbers is obtained. Then we obtain a law of iterated logarithm type result under broad conditions. The rest of the paper is arranged as follows. Section 2.2 begins with the formulation of the problem. Also presented are the numerical approximation algorithm, main conditions to be used, and some preliminary results. Section 2.3 is devoted to laws of large numbers
for certain sequences. It is shown that an appropriate average of a function of the iterates converges almost surely. The limit is precisely the ergodic mean of the underlying function (i.e., the mean of the function with respect to the invariant measure). It demonstrates that the discrete-time counterpart of the long-run average or the approximation to the average in continuous time has the correct ergodic limit. Continuing our investigation, Section 2.4 further addresses the convergence rate issue. Our effort in this section is to derive a sharp error bound in the form of law of iterated logarithm. A couple of lemmas are proved in this section, where as the proof of the main convergence rate result is postponed to Section 2.4. Finally, a couple of examples are given in Section 2.5 for demonstration purposes together with some further remarks.
2.2
Formulation and Preliminary Results
2.2.1 Formulation
Let (Ω, P,F) be a probability space. Consider anr-dimensional switching diffusion pro-cess given by
dX(t) =b(X(t), α(t))dt+σ(X(t), α(t))dW(t), (2.1)
where b(·,·) and σ(·,·) are appropriate Rr-valued and Rr×d-valued functions representing the drift and diffusion coefficients, respectively, W(·) is a d-dimensional standard Brown-ian motion, and {α(t)} is a continuous-time Markov chain with finite state space M =
{1,2, . . . , m0}and generator Q= (qij) such thatα(·) is independent of the Brownian motion
W(·). Write qeij =|qij|. For any g(·,·) :Rr× M →R, whose partial derivatives with respect
switching diffusion by Lg(x, ι) = ∇g′(x, ι)b(x, ι) + tr(∇2g(x, ι)A(x, ι)) +Qg(x,·)(ι) = r ∑ i=1 bi(x, ι) ∂g(x, ι) ∂xi + 1 2 r ∑ i,j=1 aij(x, ι) ∂2g(x, ι) ∂xi∂xj +Qg(x,·)(ι), ι∈ M, (2.2)
where ∇g(·, ι) and∇2g(x, ι) denote the gradient and Hessian of g(·, ι), respectively,
Qg(x,·)(ι) = m0 ∑ ℓ=1 qιℓg(x, ℓ), and A(x, ι) = (aij(x, ι)) =σ(x, ι)σ′(x, ι)∈Rr×r,
and z′ denotes the transpose of z.
Next we construct an approximation numerical algorithm. Suppose {γn, n ≥ 1} is a
decreasing sequence of positive real numbers satisfying γn ↓ 0 and Γn =
∑n
k=1γk ↑ ∞
as n → ∞. Choosing arbitrary initial data x0, we consider the following approximation
algorithm:
xn+1 =xn+γn+1b(xn, αn) +√γn+1σ(xn, αn)Un, (2.3)
where αn = α(Γn) and {Un} is a sequence of i.i.d. random vectors with EU1 = 0, EU1U1′ =
Ir×r, and E|U1|2p <∞ for a positive constant pto be specified in Assumption (H). We also
assume that {Un, n ≥ 1} is independent of {α(t), t ≥ 0}. It may be called a skeleton of
the continuous-time Markov chainα(t). Comparing to the algorithm given in [38], there are
three main differences. First, in lieu of Brownian motion increments, an i.i.d. sequence is used to approximate for the approximation in (2.3). Second, a discrete-time Markov chain is used in [38], whereas a skeleton of the continuous Markov chain is used in the current paper. Finally, the main objective of [38] is to obtain the convergence to the stationary measure,
whereas in the current paper, we focus on convergence rate issue. Throughout this paper, we assume the following assumption holds.
Assumption (H). Assume thatpis a fixed constant satisfyingp≥4. For eachi∈ M, there exists a C2 function V(·, i) :Rr 7→R for each i∈ M, with inf
x,iV(x, i)>0 satisfying
(i) |∇2V(x, i)|is bounded uniformly and lim|x|→∞V(x, i) = ∞;
(ii) for anyx∈Rr and i∈ M,|∇V(x, i)|2+|σ′(x, i)σ(x, i)|+|b(x, i)|2 ≤cV(x, i) for some constant c >0;
(iii) for any x∈Rr and i∈ M,
pVp−1(x, i) ( ∇V′(x, i)b(x, i) +θp(i)tr[σ′(x, i)σ(x, i)] ) +QVp(x,·)(i)≤ −λVp(x, i) +β 2V(x, i) ( ∇V′(x, i)b(x, i) +θ2(i)tr[σ′(x, i)σ(x, i)] ) +QV2(x,·)(i)≤ −λV2(x, i) +β
for some λ >0 andβ ∈R, where for q >1,
θq(i) :=
1 2xsup∈Rr
e
θ∇2V(x,i)+(q−1)(∇V(x,i)∇V′(x,i))/V(x,i),
with θeB = max{ω1, . . . , ωr,0} for an r× r symmetric matrix B having eigenvalues
ω1, . . . , ωr;
(iv) |b(x1, i)−b(x2, i)|+|σ(x1, i)−σ(x2, i)| ≤ K|x1 −x2| for any x1, x2 ∈ Rr and some
(v) there exists a κ >0 such that for any ξ∈Rr,
ξ′σ(x, i)σ′(x, i)ξ ≥κ|ξ|2.
Remark 2.1. (1) Using Assumption (H), one can conclude that (X(t), α(t)) in (2.1) has a unique invariant measure. In what follows, we write the invariant distribution as ν(·,·) = (ν(·, i) :i∈ M) and the associated invariant density as µ(·,·) = (µ(·, i) :i∈ M).
(2) By (iii), one can verify that there exists κ2 ≥1 such that for sufficiently large |x|,
max
i V(x, i)≤κ2mini V(x, i).
2.2.2 Preliminary Results
Denote by (Xx(t), αi(t)) the switching diffusion with initial dataX(0) = xandα(0) =i.
In this paper, we concern ourselves with regular switching diffusion processes. Recall that (Xx(t), αi(t)) is regular if and only if
β∞ = lim
n→∞βn =∞a.s., (2.4)
where
βn:= inf{t:|Xx(t)|=n}.
That is, the process is regular if it does not have finite explosion time a.s. Set Ue :=D×J
with J ⊂ M and D ⊂ Rr being an open set with compact closure. Set τx,i
e
U = inf{t :
(Xx(t), αi(t))∈Ue}.A regular process (Xx(·), αi(·)) is recurrent with respect toUe ifP{τex,i
∞} = 1 for any (x, i) ∈ Dc× M, where Dc denotes the complement of D; otherwise, the
process is transient with respect to U. For a recurrent process, if the expectation of its
recurrence time is finite (i.e., EτUx,i <∞for some setUe =D×J, whereJ ⊂ M andD⊂Rr
is a bounded open set with compact closure), it is said to be positive recurrent w.r.t. Ue; otherwise, the process is null recurrent w.r.t. Ue.
Remark 2.2. In [64, Theorem 4.4], we have proved the following assertion. Suppose that (Xx(t), αi(t)) is positive recurrent. Then the following results hold.
(a) The process has a unique stationary distributionν(·,·).
(b) Denote by µ(·,·) the unique stationary density associated with the stationary distri-bution ν(·,·). Suppose that ψ : Rr × M 7→ R is a Borel measurable function such
that ∑ ι∈M ˆ Rr |ψ(x, ι)|µ(x, ι)dx <∞. Then asT → ∞, 1 T ˆ T 0 ψ(X(t), α(t))dt→ψ = ∑ ι∈M ˆ Rr ψ(x, ι)µ(x, ι)dx a.s. (2.5)
Remark 2.3. In what follows, our rate of convergence study will be through the consider-ation of ψ(·,·). Moreover, we shall assume that the system of equations
with
ψ = ∑
ι∈M
ˆ
ψ(x, ι)µ(x, ι)dx= 0, (2.7)
has a unique solution, where ψ(·, ι) is a continuous function for each ι ∈ M. Sufficient conditions can be devised. However, these are not the primary concern of the current paper. Rather, we start with assuming the existence of the unique solution above. In addition, in view of Remark 2.2, in lieu of working with ´0T ψ(Xx(t), αi(t))dt, we shall work with
1
T
ˆ T
0
Lϕ(Xx(t), αi(t))dt. (2.8)
Furthermore, we are in fact, working with the discrete-time counterpart of (2.8). It will become clear in the following sections.
2.3
Approximation of Ergodic Means
In this section, we show that under suitable conditions, the numerical approximation sequence {xn, αn} possesses the strong law of large numbers in the almost sure sense. The
main result is stated in Theorem 2.12. It shows that an appropriate average of the iterates obtained in the numerical algorithm converges to the mean with respect to the invariant measure. The limit is precisely the ergodic average given in (2.5). It thus confirms that the numerical approximation also has the desired ergodicity as stated in Remark 2.2. To obtain the result, we first state several preparatory results. Then the ergodicity is derived for the discrete approximation.
Lemma 2.4. Let {ξn} be a sequence of positive random variables. Then
∑∞
if
∞
∑
n=1
Eξn<∞.
Note that Lemma 2.4 is known as Levy’s lemma.
Lemma 2.5. [15, Theorem 2.18] Let{an,An}be a martingale difference sequence satisfying
that for some 1≤q ≤2,
∞ ∑ n=1 1 ΛqnE ( |an|qAn−1 ) <∞ a.s.,
where {Λn} is positive and Λn↑ ∞. Then we have
lim N→∞ 1 ΛN N ∑ n=1 an = 0 a.s.
Remark 2.6. In view of Lemma 2.4, the condition in the above lemma can be replaced by
∞ ∑ n=1 1 ΛqnE| an|q <∞ a.s. for 1≤q≤2.
Denote byFnandGntheσ-algebras generated by{U1, . . . , Un−1,α1, . . . , αn}and{U1, . . .,
Un, α1, . . . , αn}, respectively. Write En(·) = E(·|Fn) andEen(·) =E(·|Gn).
Lemma 2.7. We have EnI(αn+1 =j) = EenI(αn+1 =j) = (1 +o(1))γn+1 ∑ i∈M qijI(αn =i) +I(αn=j).
Proof.Since α(t) and Un are independent and α(t) is a Markov process, EnI(αn+1 =j) = EenI(αn+1 =j) =E[I(αn+1 =j)|αn] = (1 +o(1))γn+1 ∑ i∈M qijI(αn =i) +I(αn=j).
Lemma 2.8. Under Assumption (H), we have
sup
n E
Vp(xn, αn)<∞. (2.9)
Proof.Write ∆xn+1 =xn+1−xn. Note that
EnVp(xn+1, αn+1)−Vp(xn, αn)
=En[Vp(xn+1, αn+1)−Vp(xn+1, αn)]
+En[Vp(xn+1, αn)−Vp(xn, αn)].
For the second term, there existsξn+1 on the line segment joiningxn+1and xnsuch that
Vp(x
n+1, αn) = Vp(xn, αn) + (∇Vp)′(xn, αn)∆xn+1+ 12(∆xn+1)′∇2Vp(ξn+1, αn)∆xn+1.
Noting that
by the definition of θp, we have
(∆xn+1)′∇2Vp(ξn+1, αn)∆xn+1 ≤2pθp(αn)Vp−1(ξn+1, αn)|∆xn+1|2
Because |∇√V(x, i)| is bounded and using the inequality (u+v)q ≤uq+c(uq−1v +vq) for
q≥2, we have Vp−1(ξn+1, αn) = (√ V(ξn+1, αn) )2(p−1) ≤(√V(xn, αn) +c|∆xn+1| )2(p−1) ≤Vp−1(xn, αn) +c ( Vp−3/2(xn, αn)|∆xn+1|+|∆xn+1|2(p−1) ) ≤Vp−1(xn, αn) +c√γn+1Vp−1(xn, αn)(1 +|Un|2p−2). Therefore, we have (∆xn+1)′∇2Vp(ξn+1, αn)∆xn+1 ≤2pθp(αn)Vp−1(xn, αn)|∆xn+1|2 +cγ3n/+12Vp(xn, αn)(1 +|Un|2p). (2.10) Since En|∆xn+1|2 =γn+1tr[σ′(xn, αn)σ(xn, αn)] +γn2+1b′(xn, αn)b(xn, αn), we have En[Vp(xn+1, αn)−Vp(xn, αn)] ≤pγn+1Vp−1(xn, αn){(∇V)′(xn, αn)b(xn, αn) +θp(αn)tr[σ′(xn, αn)σ(xn, αn)]} +o(γn+1)Vp(xn, αn).
For the first term, noting that ∇√V(x, i) is bounded, for 0< s <1, |∇V′(xn+s∆xn+1, i)| ≤ √ V(xn+s∆xn+1, i) ≤√V(xn, i) +c|∆xn+1| ≤c√V(xn, αn)(1 +|Un|), therefore, En[(∇Vp)′(xn+s∆xn+1, αn+1)−(∇Vp)′(xn+s∆xn+1, αn)]∆xn+1 =En{Een[(∇Vp)′(xn+s∆xn+1, αn+1)−(∇Vp)′(xn+s∆xn+1, αn)]∆xn+1} =∑ i̸=j En{[(∇Vp)′(xn+s∆xn+1, j)−(∇Vp)′(xn+s∆xn+1, i)]∆xn+1I(αn=i)EenI(αn+1 =j)} =O(γn+1)En ∑ i̸=j [(∇Vp)′(xn+s∆xn+1, j)−(∇Vp)′(xn+s∆xn+1, i)]∆xn+1qijI(αn =i) =O(γn3/+12)Vp(xn, αn). We have En[Vp(xn+1, αn+1)−Vp(xn+1, αn)] =En[Vp(xn, αn+1)−Vp(xn, αn)] +En ˆ 1 0 [(∇Vp)′(xn+s∆xn+1, αn+1)−(∇Vp)′(xn+s∆xn+1, αn)]∆xn+1ds =γn+1QVp(xn,·)(αn) +o(γn+1)Vp(xn, αn).
Thus, for some λ >0,
EnVp(xn+1, αn+1)−Vp(xn, αn)≤ −λγn+1Vp(xn, αn) +o(γn+1)(1 +Vp(xn, αn)). (2.11)
Then we have
By induction, we conclude that (2.9) holds.
Remark 2.9. Using the same argument, we can verify that
|V2(xn+1, i)−V2(xn, i)| ≤c
√
γn+1V2(xn, i)(1 +|Un|4). (2.12)
Lemma 2.10. For each i∈ M and C2 functions f(·, i) :Rr →R, set
T(x, i) =σ′(x, i)∇2f(x, i)σ(x, i) H(x, i) = (∇f(x, i))′b(x, i) + 1 2tr[T(x, i)] R(y, x, i) =f(y, i)−f(x, i)−(∇f(x, i))′(y−x)− 1 2(y−x) ′∇2 f(x, i)(y−x). Then − N∑−1 n=0 γn+1Lf(xn, αn) =− N∑−1 n=0 γn+1 ( H(xn, αn) +Qf(xn,·)(αn) ) = N−1 ∑ n=0 10 ∑ i=1 Ii,n, where I1,n =f(xn+1, αn+1)−f(xn, αn) I2,n =√γn+1(∇f)′(xn, αn)σ(xn, αn)Un I3,n = 1 2γn+1 [ Un′T(xn, αn)Un−tr[T(xn, αn)] ] I4,n =γ 3/2 n+1b′(xn, αn)∇2f(xn, αn)σ(xn, αn)Un I5,n = 1 2γ 2 n+1b′(xn, αn)∇2f(xn, αn)b(xn, αn) I6,n =R(xn+1, xn, αn+1) I7,n = [∇f′(xn, αn+1)− ∇f′(xn, αn)]∆xn+1 I8,n = 1 2∆x ′ n+1[∇ 2 f(xn, αn+1)− ∇2f(xn, αn)]∆xn+1 I9,n =−{f(xn, αn+1)−Enf(xn, αn+1)} I10,n =o(1)γn+1Qf(xn,·)(αn).
Proof.Using Taylor’s expansion, f(xn+1, αn+1)−f(xn, αn+1) =∇f′(xn, αn)∆xn+1+ 1 2∆x ′ n+1∇ 2 f(xn, αn)∆xn+1+R(xn+1, xn, αn+1) +[∇f′(xn, αn+1)− ∇f′(xn, αn)]∆xn+1 +1 2∆x ′ n+1[∇ 2 f(xn, αn+1)− ∇2f(xn, αn)]∆xn+1 =√γn+1∇f′(xn, αn)σ(xn, αn)Un+γn+1H(xn, αn) +1 2γn+1 [ Un′T(xn, αn)Un−tr[T(xn, αn)] )] +γn3/+12b′(xn, αn)∇2f(xn, αn)σ(xn, αn)Un +1 2γ 2 n+1b′(xn, αn)∇2f(xn, αn)b(xn, αn) +R(xn+1, xn, αn+1) +[∇f′(xn, αn+1)− ∇f′(xn, αn)]∆xn+1 +1 2∆x ′ n+1[∇ 2f(x n, αn+1)− ∇2f(xn, αn)]∆xn+1.
Noting that Enf(xn, αn+1) =f(xn, αn) + [1 +o(1)]γn+1Qf(xn,·)(αn), we have N∑−1 n=0 ( f(xn+1, αn+1)−f(xn, αn+1) ) =f(xN, αN)−f(x0, α0) − N∑−1 n=0 ( f(xn, αn+1)−f(xn, αn) ) =f(xN, αN)−f(x0, α0) − N∑−1 n=0 ( f(xn, αn+1)−Enf(xn, αn+1) ) −(1 +o(1))Qf(xn,·)(αn).
For the rest of this section, we also assume that the step-size {γn} satisfies ∞ ∑ n=1 γn Γ2 n <∞ and ∞ ∑ n=1 γ 3 2 n Γn <∞.
Lemma 2.11. Under Assumption (H), we have
lim sup N→∞ 1 ΓN N∑−1 n=0 γn+1V2(xn, αn)<∞ a.s., (2.13) and lim sup N→∞ V2(x N, αN) ΓN <∞ a.s.
Proof.Similar to the proof of Lemma 2.8, for some ¯λ >0 and ¯β,
EnV2(xn+1, αn+1)≤V2(xn, αn) +γn+1( ¯β−λV¯ 2(xn, αn)), we have V2(xn, αn)≤ ¯ β ¯ λ + 1 γn+1λ¯ [ V2(xn, αn)−EnV2(xn+1, αn+1) ] ,
and 1 ΓN N∑−1 n=0 γn+1V2(xn, αn) ≤ ¯ β ¯ λ + 1 ΓNλ¯ N∑−1 n=0 [ V2(xn, αn)−EnV2(xn+1, αn+1) ] = ¯ β ¯ λ + 1 ΓNλ¯ N∑−1 n=0 [ V2(xn+1, αn+1)−EnV2(xn+1, αn+1) ] + 1 ΓN¯λ { V2(x0, α0)−V2(xN, αN) } ≤ β¯¯ λ + 1 ΓNλ¯ N∑−1 n=0 [ V2(xn+1, αn+1)−EnV2(xn+1, αn+1) ] + 1 ΓN¯λ V2(x0, α0). (2.14)
By (2.12) and noting that
En{V2(xn, αn+1)−V2(xn, αn)}2 = (1 +o(1))γn+1 ∑ i̸=j qij[V2(xn, j)−V2(xn, i)]2I(αn =i), we have E[V2(xn+1, αn+1)−EnV2(xn+1, αn+1)]2 ≤cE[V2(xn+1, αn+1)−V2(xn, αn)]2 ≤c{E{V2(xn+1, αn+1)−V2(xn, αn+1)}2 +E{V2(xn, αn+1)−V2(xn, αn)}2 } ≤cγn+1EV4(xn, αn). Thus ∞ ∑ n=1 1 Γ2 n E(V2(xn+1, αn+1)−EnV2(xn+1, αn+1) )2 ≤c ∞ ∑ n=1 γn+1 Γ2 n <∞. By Lemma 2.5, we have lim N→∞ 1 ΓN N∑−1 n=0 [ V2(xn+1, αn+1)−EnV2(xn+1, αn+1) ] = 0 a.s.,
and lim sup N→∞ 1 ΓN N∑−1 n=0 γn+1V2(xn, αn)<∞ a.s. Also by (2.14), we have lim sup N→∞ 1 ΓNλ¯ V2(xN, αN) ≤ β¯¯ λ + lim supN→∞ { 1 ΓN¯λ V2(x0, α0) + 1 ΓNλ¯ N∑−1 n=0 [ V2(xn+1, αn+1)−EnV2(xn+1, αn+1) ]} <∞ a.s., (2.15) so lim sup N→∞ V2(xN, αN) ΓN <∞ a.s.
Theorem 2.12. For each i∈ M, let h(·, i) be a Borel measurable function such that
lim sup
|x|→∞
|h(x, i)|
V2(x, i) <∞.
Then under Assumption (H), we have
lim N→∞ 1 ΓN N∑−1 n=0 γn+1h(xn, αn) = ∑ i∈M ˆ h(x, i)ν(dx, i) a.s. (2.16)
Proof.First we show for any f(·, i)∈C3 with compact support (written as C3 0), lim N→∞ 1 ΓN N∑−1 n=0 γn+1Lf(xn, αn) = 0 a.s.
Using Lemma 2.10, by the boundedness of f, we have lim N→∞ 1 ΓN N−1 ∑ n=0 I1,n = 0 a.s. (2.17)
Because {Ii,n} for i = 2,3,4 are martingale difference sequences with respect to Fn+1
and ∞ ∑ n=1 EIi,n2 Γ2 n <∞,
we have that for i= 2,3,4,
lim N→∞ 1 ΓN N−1 ∑ n=0 Ii,n= 0 a.s. (2.18)
ForI5,n, sinceE|I5,n|=O(γn2+1) and
∑∞ n=1 E|I5,n| Γn <∞, lim N→∞ 1 ΓN N−1 ∑ n=0 I5,n = 0 a.s. (2.19)
ForI6,n, sincef(·, i) is aC03 function, ∇2f(·, i) is bounded and Lipschitz. Because
R(xn+1, xn, αn+1) ≤c|xn+1−xn|3 ≤cγ 3 2 n+1V 3 2(xn, αn)(1 +|Un|3), and ∑∞n=1 E|I6,n| Γn <∞, we have lim N→∞ 1 ΓN N−1 ∑ n=0 I6,n = 0 a.s. (2.20)
ForI7,n, noting thatEn∇f′(xn, αn+1) =Een∇f′(xn, αn+1),
I7,n = [∇f′(xn, αn+1)−Een∇f′(xn, αn+1)]∆xn+1
+(1 +o(1))γn+1Q∇f′(xn,·)(αn)∆xn+1
=:J1,n+J2,n.
Note that J1,n is a martingale difference sequence with respect to Gn+1. For any bounded
function h1 and h2, e En [ (h1(xn, αn+1)−Eenh1(xn, αn+1))(h2(xn, αn+1)−Eenh2(xn, αn+1)) ] =En[h1(xn, αn+1)h2(xn, αn+1)]−Enh1(xn, αn+1)Enh2(xn, αn+1) = (1 +o(1))γn+1 ∑ i,j∈M e qijh1(xn, j)h2(xn, j)I(αn=i), and e EnJ12,n = (1 +o(1))γn+1 ∑ i,j∈M e qij∆x′n+1∇f(xn, j)∇f′(xn, j)∆xn+1I(αn=i). (2.21) Since EJ12,n =O(γn2) and ∑∞n=1 EJ 2 1,n Γ2 n <∞, we have lim N→∞ 1 Γn N∑−1 n=0 J1,n = 0 a.s. Since E|J2,n|=O(γ 3/2 n+1) and ∑∞ n=1 E|J2,n| Γn <∞, lim N→∞ 1 Γn N∑−1 n=0 J2,n = 0 a.s.
As a result, lim N→∞ 1 Γn N∑−1 n=0 I7,n = 0 a.s.
For I8,n, similar to (2.21), we have
e En|I8,n|= 1 2Een ∑ i̸=j ∆xn+1(∇2f(xn, j)− ∇2f(xn, i))∆xn+1I(αn =i, αn+1 =j) = (1 +o(1))γn+1 2 Een ∑ i̸=j e qij∆xn+1(∇2f(xn, j)− ∇2f(xn, i))∆xn+1I(αn=i) Then by E|I8,n|=O(γn2+1) and ∑∞ n=1 E|I8,n| Γn <∞, it follows that lim N→∞ 1 ΓN N−1 ∑ n=0 I8,n = 0 a.s. (2.22)
ForI9,n, detailed calculations lead to
EnI92,n =Enf2(xn, αn+1)−[Enf(xn, αn+1)]2 = (1 +o(1))γn+1 ∑ i,j∈M e qijf2(xn, j)I(αn=i), (2.23) thus EI92,n =O(γn+1).
SinceI9,nis a martingale difference sequence with respect toFn+1, knowing that
∑∞ n=1 EI2 9,n Γ2 n < ∞, we have lim n→∞ 1 ΓN N∑−1 n=0 I9,n = 0 a.s. (2.24)
can conclude (2.16). As a result, lim N→∞ 1 ΓN N∑−1 n=0 γn+1Lf(xn, αn) = 0 a.s.
Now for functions g(·,·) defined on Rr× M, set
e νN(g) := 1 ΓN N∑−1 n=0 γn+1g(xn, αn).
Specially, if we let g be the indicator on [x, x+dx)×i, we have
e νN(dx×i) = 1 ΓN N∑−1 n=0 γn+1I{xn∈[x, x+dx), αn =i}.
From (2.13), one can conclude that eνN(dx × i) is tight, so we can extract a convergent
subsequence. Now we prove any weak limitνe∞(dx×i) of eνN(dx×i) is an invariant measure
of the Markov process with operator L. Since for each i ∈ M, Lf(·, i) is bounded and continuous for any f(·, i)∈C3
0, we have
e
ν∞(Lf) = lim
N→∞νeN(Lf) = 0 a.s. (2.25)
Now we may apply Theorem 5 in [7]. First we check L satisfies the hypothesis. Let
E =Rr× M with the metric
d((x, i),(y, j)) = |x−y|, if i=j, |x−y|+ 1, if i̸=j.
Then E is a locally compact metric space. Consider the space of C0 functions defined on
E with norm |f|E = maxi∈Msupx∈Rr|f(x, i)|. Let D(L) be the set of all C03 functions with respect to the first variable. ThenD(L) is dense in C0(E). By Ito’s formula, for any positive
real-valued f ∈ D(L) with compact support and with maximum at (x0, ι), for (X0, α0) =
(x0, ι), it is well-known that f(X(t), α(t))−f(x0, ι)−
´t
0 Lf(X(s), α(s))ds is a martingale.
Since f(X(t), α(t)) is non-increasing att = 0+, we have Lf(x0, ι)≤ 0, which shows that L
satisfies the maximum principle. Consider the sequence of ζn(x, i) = ζ(xn) for each i ∈ M,
where ζ is a C03 function from Rr to Rand ζ(0) = 1. It is easy to see that ζn(x, i) converges
to 1 pointwise as n → ∞. Note that
Lζn(x, i) = 1 n∇ζ ′(x n)b(x, i) + 1 2n2σ ′(x, i)∇2ζ(x n)σ(x, i)
converges to 0 pointwise and supn|Lζn(x, i)|E < ∞ since σ and b grow at most linearly.
Then all the hypotheses in [7, Theorem 5] are verified. By (2.25), applying [7, Theorem 5], one concludes that eν∞(dx×i) is an invariant measure of the process represented by the solution to (2.1). By the uniqueness ofν (i.e., the invariant measure of the solutions of (2.1))
and the arbitrariness of the subsequence of νeN(dx×i), one can conclude that eν∞(dx×i) is
independent of the choice of the subsequence and eν∞(IA×i) =
´
Aν(dx, i) a.s. for any Borel
set A ⊂ Rr and i ∈ M with ν(∂A, i) = 0. Then by (2.13) and using dominant convergent
theorem, (2.16) holds.
2.4
Convergence Rates for Approximation to Ergodic Means
Theorem 2.12 asserts the ergodicity for the numerical approximation sequence to the mean with respect to the invariant measure. A natural question is: What is the rate of
convergence? This section addresses the convergence rate issue. Our main result is given in Theorem 2.14, which confirms that suitably scaled sequence of the iterates of the recursive algorithm verifies the law of iterated logarithm. This result provides a sharp error bound on the approximation error to the average with respect to the invariant measure. It provides a convergence rate result for the numerical approximation to the ergodic mean. Throughout this section assume that {γn} is a positive decreasing sequence satisfying
∞ ∑ n=1 γ 3 2 n √ Γn <∞. (2.26)
Remark 2.13. Note that condition (2.26) is not difficult to verify. For example, ifγn =nδ−1
with 0< δ <1/2, the condition is satisfied.
Theorem 2.14. For each i∈ M, let φ(·, i) be a continuous function Rr →R such that
φ:= ∑
i∈M
ˆ
φ(x, i)dν(x, i) = 0.
Assume that there exists a C2 function f(·, i) for each i ∈ M, with ∇2f(·, i) bounded and Lipschitz such that
Lf(x, i) =φ(x, i), i∈ M, and that lim sup |x|→∞ |f(x, i)| V(x, i) + |∇f(x, i)| √ V(x, i) <∞.
Under Assumption(H), the following Law of iterated logarithm holds almost surely on an en-larger probability space(Ωe,P ,e Fe)on which there exists(xen,αen)that has the same distribution
as that of (xn, αn) such that lim sup n→∞ 1 √ 2ΓNlog log ΓN N−1 ∑ n=0 γn+1φ(xen,αen) ≤( ∑ i,j∈M ˆ e qijf2(x, j)ν(dx, i) )1 2 +( ∑ i∈M ˆ (∇f)′(x, i)A(x, i)∇f(x, i)ν(dx, i) )1 2 . (2.27)
Remark 2.15. The above theorem is obtained through the use of the Skorohod representa-tion theorem; see [15, p. 269]. For notarepresenta-tional simplicity, we shall not use the “tilde” notarepresenta-tion in what follows. With a slight abuse of notation, we will state that (2.27) holds without using the “tilde” notation. That is, we consider (2.27) with (xen,αen) replaced by (xn, αn).
As a preparation, we first state some lemmas. The proof of the theorem is deferred to the next section.
Lemma 2.16. Let {an} be a sequence of positive random variables satisfying
lim N→∞ 1 N N ∑ n=1 an = 1 a.s.
Then for the Brownian motion W(·), we have
lim sup
N→∞
W(∑Nn=1an)
√
2Nlog logN = 1 a.s.
Proof.See [5].
and ψ2(·, i) be an r×r-valued function with lim sup |x|→∞ |ψ1(x, i)|∞+|ψ2(x, i)|∞ V(x, i) <∞. We have lim sup N→∞ 1 √ 2ΓNlog log ΓN N∑−1 n=0 √ γn+1ψ1′(xn, αn)Un= ( ∑ i∈M ˆ ψ′1(x, i)ψ1(x, i)ν(dx, i) )1 2 a.s. (2.28) lim sup N→∞ 1 √ ΓN N∑−1 n=0 γn+1 [ Un′ψ2(xn, αn)Un−tr(ψ2(xn, αn)) ] = 0 a.s. (2.29) and lim sup N→∞ 1 √ ΓN N∑−1 n=0 γn3/+12ψ1′(xn, αn)Un= 0 a.s. (2.30)
Proof.Noting that Enψ1′(xn, αn)Un= 0,
Gn = N∑−1
n=0
√
γn+1ψ1′(xn, αn)Un is a martingale with respect to Fn+1.
By the Skorohod representation theorem [15, p. 269], there exists a Brownian motion W(·) on a larger probability space and a sequence of stopping times {τn, n ≥1} such that
{Gn, n= 1,2, . . .} ∼ {W( n ∑ i=1 τi), n= 1,2, . . .}, Enτnq ≤Cqγnq+1|ψ1|2∞q(xn, αn) for any 0≤q≤2,
and
Enτn=γn+1ψ′1(xn, αn)ψ1(xn, αn),
whereZ1 ∼Z2 denotesZ1 andZ2 having the same distribution. By Theorem 2.12, we know
lim N→∞ 1 ΓN N−1 ∑ n=0 Enτn = ∑ i∈M ˆ ψ′1(x, i)ψ1(x, i)dν(x, i) a.s. (2.31) Since ∞ ∑ n=1 Eτ2 n Γ2 n ≤c ∞ ∑ n=1 γ2 n+1 Γ2 n E|ψ1|4∞(xn, αn) <∞, we have lim N→∞ 1 ΓN N∑−1 n=0 {τn−Enτn}= 0 a.s. (2.32)
Consequently from (2.31) and (2.32), we have
lim N→∞ 1 ΓN N∑−1 n=0 τn= ∑ i∈M ˆ ψ1′(x, i)ψ1(x, i)ν(dx, i) a.s.
By Lemma 2.16, we have proved (2.28).
Note that {γn+1[Un′ψ2(xn, αn)Un−tr(ψ2(xn, αn))]} is a martingale difference sequence
with respect to Fn+1 and
En{γn+1[Un′ψ2(xn, αn)Un−tr(ψ2(xn, αn))]}
2
Knowing that ∑∞n=1 γ
2
n+1
Γn <∞, we can conclude (2.29). Also (2.30) follows from
lim N→∞E N−1 ∑ n=1 γn3/+12 √ Γn |ψ′1(xn, αn)Un|<∞.
The proof is complete.
Proof of Theorem 2.14:By Lemma 2.11 and noting ΓN ↑ ∞, we have
lim N→∞ 1 √ ΓN log log ΓN N∑−1 n=0 I1,n = 0 a.s. (2.33) By Lemma 2.17, we have lim sup N→∞ 1 √ 2ΓNlog log ΓN N∑−1 n=0 I2,n =( ∑ i∈M ˆ (∇f)′(x, i)A(x, i)∇f(x, i)ν(dx, i) )1 2 a.s., (2.34) lim sup N→∞ 1 √ ΓNlog log ΓN N∑−1 n=0 I3,n= 0 a.s., (2.35) and lim sup N→∞ 1 √ ΓNlog log ΓN N∑−1 n=0 I4,n = 0 a.s. (2.36) Since E|I5,n|=O(γn2+1) and ∑∞ n=1 E√|I5,n| Γn <∞we have lim N→∞ 1 √ ΓN N∑−1 n=0 I5,n = 0 a.s. (2.37)
ForI6,n, since∇2f(·, i) is bounded and Lipschitz, R(xn+1, xn, αn+1) ≤c|xn+1−xn|3 ≤cγ 3 2 n+1V 3 2(xn, αn)(1 +|Un|3), and ∑∞n=1 E√|I6,n| Γn <∞, we have lim N→∞ 1 √ ΓN N∑−1 n=0 I6,n = 0 a.s. (2.38)
ForI7,n, note that{J1,n,Gn+1}is a martingale difference sequence. SinceEJ12,n =O(γn2+1)
and ∑∞n=1 EJ 2 1,n Γn <∞, we have lim N→∞ 1 √ ΓN N∑−1 n=0 J1,n = 0 a.s. Since E|J2,n|=O(γ 3/2 n+1) and ∞ ∑ n=1 E√|J2,n| Γn <∞, we have lim N→∞ 1 √ ΓN N∑−1 n=0 J2,n = 0 a.s. As a result, lim N→∞ 1 √ ΓN N∑−1 n=0 I7,n = 0 a.s. (2.39) For I8,n, since E|I8,n|=O(γn2+1) and ∞ ∑ n=1 E√|I8,n| Γn <∞,
lim N→∞ 1 √ ΓN N∑−1 n=0 I8,n = 0 a.s. (2.40)
Since {I9,n} is a martingale difference sequence with respect to Fn+1, by the Skorohod
representation theorem [15, p. 269], there exists a Brownian motion on a larger probability space and a sequence of stopping times {ςi} such that
{ n ∑ i=1 I9,i, n= 1,2, . . .} ∼ {W( n ∑ i=1 ςi), n= 1,2, . . .}, and that Enςnq ≤cEn|I9,n|2q for any 0≤q ≤2, and Enςn = (1 +o(1))γn+1 ∑ i,j∈M ˆ e qijf2(xn, j)I(αn=i). As a result, by Lemma 2.12, lim N→∞ 1 ΓN N−1 ∑ n=0 Enςn= ∑ i,j∈M ˆ e qijf2(x, j)ν(dx, i) a.s. Noting that En|I9,n|4 ≤cEn(f(xn, αn+1)−f(xn, αn))4 ≤c∑ i̸=j (f(xn, j)−f(xn, i))4EnI(αn+1 =j, αn =i) ≤cγn+1 ∑ i̸=j e qij(f(xn, j)−f(xn, i))4I(αn=i),
and N∑−1 n=0 1 Γ2 n Eςn2 ≤c N∑−1 n=0 1 Γ2 n E|I9,n|4 ≤c N∑−1 n=0 γn+1 Γ2 n <∞, it follows that lim N→∞ 1 ΓN N∑−1 n=0 {ςn−Enςn}= 0 a.s. Therefore, we have lim N→∞ 1 ΓN N∑−1 n=0 ςn= ∑ i,j∈M ˆ e qijf2(x, j)ν(dx, i) a.s. which leads to lim sup n→∞ 1 √ 2ΓNlog log ΓN N∑−1 n=0 I9,n =( ∑ i,j∈M ˆ e qijf2(x, j)ν(dx, i) )1 2 a.s. (2.41)
Since I10,n can be eliminated in the Lf(xn, αn), (2.33)-(2.41) lead to
lim sup n→∞ 1 √ 2ΓNlog log ΓN N∑−1 n=0 γn+1φ(xn, αn) ≤( ∑ i,j∈M ˆ e qijf2(x, j)ν(dx, i) )1 2 +( ∑ i∈M ˆ (∇f)′(x, i)A(x, i)∇f(x, i)ν(dx, i) )1 2 a.s. (2.42)
The proof is complete.
2.5
Examples and Remarks
In this section, we demonstrate the convergence rates in terms of law of iterated loga-rithm by looking at two simple examples.
2.5.1 Examples
Example 2.18. Suppose that α(t) is a 2-state Markov chain with generator
Q= −16 16 4 −4 (2.43)
and state space M = {1,2}, and B(·) is a 1-dimensional standard Brownian motion inde-pendent of α(·). Consider a switching diffusion given by (2.1), where
b(x,1) = 1 10 √ x2+ sin2x, b(x,2) =− x3 1 +x2, σ(x,1) = |x|+ 1 4 , and σ(x,2) = x2+ 1 4|x|+ 1. Let V(x,1) =x2+ 1, V(x,2) = x 2 21/4 + 1 and p= 4.
Detailed calculation shows that Assumption (H) holds. Letf(x, i) =x. Then we have
φ(x,1) = √ x2 + sin2x 10 , φ(x,2) =− x3 1 +x2.
We use the decreasing step-sizes γn =n−0.6. Write
Cn = 1 √ 2Γnlog log Γn n−1 ∑ k=0 γk+1φ(xk, αk).
Using a sequence of i.i.d random variables {Ui} with standard normal distribution to
ap-proximate the Brownian motion in (2.1). To demonstrate, we provide a sketch of the graph ofCn verse Γn(plotting Cnas a function of Γn) in Figure 1. From the graph, it is easily seen
that the curve Cn is bounded, which verifies the conclusion of Theorem 2.14. That is, the
convergence rate to the invariant measure is of the form of the law of iterated logarithm.
Figure 1: Cn as a function of Γn
Example 2.19. Suppose thatα(t) is a 2-state Markov chain with the same generator given by (2.43) and state spaceM={1,2}, andB(·) is a 1-dimensional standard Brownian motion independent of α(·). Consider a switching diffusion given by (2.1), where
b(x,1) = 1 10x, b(x,2) =−x, σ(x,1) = √ 1 +x2 15 , and σ(x,2) = 1 +|x| √ 15 . Let V(x,1) =x2+ 1, V(x,2) = x 2 21/4 + 1 and p= 4.
Different from Example 2.18, let f(x, i) =x2. Then we have φ(x,1) = 4 15x 2+ 1 15, φ(x,2) =−2x 2+(1 +|x|)2 15 .
We still use the decreasing step-sizes γn =n−0.6 and carry out N = 106 recursions by using
a sequence of i.i.d random variables {Ui} with standard normal distribution. The curve Cn
is plotted as a function of Γn in Figure 2, which is seen to be bounded. The boundedness
of Cn demonstrates the conclusion of Theorem 2.14 again. Especially, we note that Cn may
decrease only when αn= 2.
2.5.2 Remarks
This paper has focused on decreasing step-size algorithms. In lieu of such algorithms, we could also examine a constant step-size algorithm of the form
xn+1 =xn+εb(xn, αn) +
√
εσ(xn, αn)Un, (2.44)
where ε > 0 is a small parameter serving as the step-size, αn = α(εn) with α(·) being the
continuous-time Markov chain, and {Un} is a sequence of i.i.d. random variables as before.
Define a piecewise constant interpolation as xε(t) = x
n for t ∈[nε, nε+ε). Then it can be
shown that xε(·) converges weakly to the solution of the switching diffusion process (2.1). The convergence to the invariant measure can be studied by examining the distribution of
xε(·+t
ε) for any tε → ∞ as ε→0. Equivalently, we look at the asymptotic distribution of
the underlying process as ε →0, n→ ∞, and εn→ ∞. A discrete version of the of the law of large numbers corresponding (2.5) can be defined. Furthermore, the corresponding rates of convergence can be studied.
In our study, the independence of αn and Un plays an important role. The current
approach may not be easily adopted to treat convergence rates of ergodic means for switching diffusions with x-dependent switching [63]. Thus new methods are needed for examining the
CHAPTER 3 STOCHASTIC DELAY SYSTEMS WITH
INFINITE DELAY
3.1
Introduction
Because time delays are ubiquitous, pervasive, and entrenched in everyday life, they have received considerable attention. In the context of dynamic systems, a class of such systems, namely, functional differential equations has become an important focal point of research and investigation. The motivation stems from non-instant transmission phenomena, for example, high velocity fields in wind funnel experiments, or other memory processes, or specially biological motivations (see [14, 31, 33, 49]) such as species’ growth or incubating time on disease models among many others. Theory of functional differential equations with infinite delay and its applications were established and developed in the 1970s and 1980s; see [6, 16–19, 29, 50, 51] and references therein. A comprehensive theory of functional differential equations with infinite delay can be found in [19]. Recently, Theories of functional differential equations with infinite delay including stability and their applications have attracted much of researchers’ attention; see e.g., [1,4,6,9,11,14,31,39,58]. Because uncertainties are commonly encountered in many real systems and are often sources of instability [30], much work has been devoted to SFDEs and their applications, for example, [2,3,30,35,36,39,45,46,54]. Along this line, theory of SFDEs with infinite delay has also received more and more attentions, for example, [20, 53, 57, 59, 60].
It is well-known that the solutions of stochastic functional or delay differential equations are non-Markov since they depend on their history, so none of the properties of solutions based on the Markov property cannot be examined. To overcome this difficulty, Mohammed
[46] examined solution maps of SFDEs with finite delay appropriate phase spaces and proved that the solution maps have Markov property. Based on the Markov property of solution maps of SFDEs with finite delay, Bao et. al. [2, 3] examined the ergodicity. Since distributions of the solutions are the marginal distributions of that of the solution maps, the existence of stationary distributions of the solution maps implies that of the solutions.
In fact, the solution maps possess many nice properties for delay systems, for example, continuous semigroup of transformations; see [6, 16–19, 29, 50, 51]. However, as mentioned in [28], of fundamental importance for all approaches is the right choice of the phase space which in most cases is a Banach space of functions or of equivalence classes of functions. For functional equations with finite delays, this is generally not a difficult problem. But for infinite delay equations the choice of an appropriate phase space is non-trivial; see [48].
Let us consider a SFDE with infinite delay of the form
dx(t) =f(xt)dt+g(xt)dw(t) (3.1)
on t ≥ 0 with the initial data x0 = ξ ∈ Cr, where xt = xt(θ) =: {x(t+θ),−∞ < θ ≤ 0},
f :Cr →Rn and g :Cr →Rn×m are Borel measurable, w(t) is an m-dimensional Brownian
motion. To show the dependence of the solution x(t) on the initial data, we also write x(t)
as x(t;ξ) or x(t;t0, ξ) if the initial segment is xt0 =ξ at t0. Correspondingly, we also write
xt as xt(ξ) or xt(t0, xt0). When −∞ < θ ≤ 0 is considered, xt(θ) can be written as xt(θ;ξ)
orxt(θ;t0, ξ) if the initial segment is ξ att0. If (3.1) has a solutionx(t;t0, ξ) with the initial
segment ξ at t0, then xt(t0;ξ) is called the solution map. In this paper, aiming at
solutions from different initial data, and existence of invariant measure of the solution map, we choose the phase space with the fading memory to be Cr defined as follows: for a given
r >0, Cr = { φ∈C((−∞,0];Rn) : lim θ→−∞e rθφ(θ) exists inRn}, (3.2)
whereC((−∞,0];Rn) is the family of continuous functions from (−∞,0] toRn; see Appendix
A and reference [19] for more details on this phase space and its properties. Our contributions of this paper are as follows.
(i) Existence and uniqueness of the solutions of the functional stochastic differential equa-tions with infinite delay are examined.
(ii) We obtain mean squares estimates as well as estimates of difference of solutions on large-time with different initial data for the solutions of the SFDEs.
(iii) Because the solutions of the SFDEs are not Markov, a viable alternative for studying further asymptotic properties is to use solution maps or segment processes. By exam-ining solution maps, we investigate the Markov properties as well as strong Markov properties. Also obtained are adaptivity (measurability) and continuity, mean-square boundedness, and convergence of solution maps from differential initial data. It should be noted that although Mohammed [46] established the continuity, adaptivity, and Markov property of the solution map for SFDEs with finite delay, little is known re-garding these properties for SFDEs with infinite delay to the best of our knowledge.
(iv) We establish ergodicity of the underlying processes and establishes existence of the invariant measure for SFDEs with infinite delay.
The rest of the paper is arranged as follows. Section 3.2 provides necessary notation, assumptions, and lemmas for preparation of our study. Section 3.2 examines the existence and uniqueness of the solution of (3.1) and its asymptotic properties including the mean-square boundedness and convergence of different solutions from different initial data. By the properties of the solution x(t) and the phase space Cr, Section 3.3 investigates properties
of the solution map xt including continuity and adaptivity as well as the strong Markov
property. To establish existence and uniqueness of invariant measure of the solution map, this section also examines the mean-square boundedness and large-time estimates from different initial data of the solution map. Based on these properties, Section 3.5 establishes existence and uniqueness of invariant measure of the solution map. Finally, the last section provides certain properties of the phase space Cr.
3.2
Formulation
Throughout this paper, unless otherwise specified, we use the following notation. Let (Ω,F,Ft,P) be a complete probability space with a filtration {Ft}t≥0 satisfying the usual
conditions (i.e., it is right continuous and increasing while F0 contains all P-null sets). Let
w(t) be an m-dimensional Brownian motion defined on this probability space. In this paper,
without loss of generality, let us choose Ft = σ{w(s) : 0 ≤ s ≤ t}, which is the natural
filtration generated by the Brownian motionw(t). Ifx(t) is anRn-valued stochastic process,
definext=xt(θ) :={x(t+θ) :−∞< θ≤0}fort ≥0. Let1A denote the indicator function
of the set A. Let | · | be the Euclidean norm in Rn. If A is a vector or matrix, its transpose
is denoted by A′. If A is a matrix, denote its trace norm by |A|=√trace(A′A). The inner product ofX, Y ∈Rn is denoted byX′Y. LetRn denote the n-dimensional Euclidian space.
