Fully coupled forward backward stochastic differential equations on Markov chains

R E S E A R C H

Open Access

Fully coupled forward-backward

stochastic differential equations

on Markov chains

Shaolin Ji

_{, Haodong Liu}

_{and Xinling Xiao}

*_{Correspondence:}

[email protected]

2_{School of Mathematical Sciences,}

Shandong Normal University, Jinan, 250014, P.R. China

Full list of author information is available at the end of the article

Abstract

We define fully coupled forward-backward stochastic differential equations on spaces related to continuous-time finite-state Markov chains. Existence and uniqueness results of the fully coupled forward-backward stochastic differential equations on Markov chains are obtained.

Keywords: forward-backward stochastic differential equations; monotone assumption; Markov chain

1 Introduction

Since the first introduction by Pardoux and Peng [] in , the theory of nonlinear back-ward stochastic differential equations (BSDEs) driven by a Brownian motion has been in-tensively researched by many researchers and has achieved abundant theoretical results. Now this theory is a powerful tool in stochastic analysis. It also has many important appli-cations, namely in stochastic control, stochastic differential games, finance, and the theory of partial differential equations (PDEs).

In the classic BSDE theory, we consider a Brownian motion as the driver, but a Brownian motion is a kind of very idealized stochastic model, which limits greatly the applications of the classic BSDEs. There are many results about BSDEs associated with jump process. Tang and Li [] first discussed BSDEs driven by a Brownian motion and Poisson process; Nualart and Schoutens [] considered BSDEs driven by a Brownian motion and Lévy pro-cess. Furthermore, there are also results where the Brownian motion in the diffusion term of the BSDE is replaced by another process. For example, Cohen and Elliott [] studied BSDEs driven by a continuous-time finite-state Markov chain. After that, many results, such as a comparison theorem about this kind of BSDEs, nonlinear expected results [, ], and so on, appeared.

Along with the rapid development of the BSDE theory, the theory of fully coupled forward-backward stochastic differential equations (FBSDEs), closely related to BSDEs, has been developed rapidly. Fully coupled FBSDEs with Brownian motion can be encoun-tered in the optimization problem when applying stochastic maximum principle (see []) and in mathematical finance when considering a large investor in security market (see []). Such FBSDEs are also used in the potential theory (see []). As we know now, to get the

existence and uniqueness results of fully coupled FBSDEs solutions, there are mainly three methods: the method of contraction mappings [, ], the four-step scheme [], and the method of continuation [, ]. For more details on fully coupled FBSDEs, we refer to Yong [], Maet al.[], or Briand and Hu [] and the references therein.

In this paper, we study fully coupled FBSDEs driven by a martingale generated by a continuous-time finite-state Markov chain. Inspired by Peng and Wu [], we introduce anm×nfull-rank matrixGto overcome the problem caused by the different dimensions of SDE and BSDE. Using the method of continuation, the Itô product rule of semimartin-gales, and the fixed point principle, based on the theory of BSDEs driven by a continuous-time finite-state Markov chain, we obtain existence and uniqueness results of the FBSDEs on a Markov chain. It is worth pointing out that due to the property of the martingale gen-erated by a finite-state Markov chain, the form of the monotone assumptions we employed here is different from that in Peng and Wu [].

Recently, many works have been done on BSDEs with Markov chains. In fact, there are two major formulations of the state space of the Markov chain in the literature. One is the set of unit vectors in some Euclidean space []. The other one is the set of positive inte-gers []. Each of the formulations was used by many researchers when they studied BS-DEs driven by Markov chains. When a martingale representation was needed, the second formulation was necessary. The first one facilitates the mathematics, that is, the symbol-ism is nicer; the second one makes the mathematics more rigorous. Given a continuous-time Markov chain, assuming its state space be the set of positive integers, there exists an integer-valued random measure that counts the jumps of a Markov chain. Crepey and Ma-toussi [] and Crepey [] considered BSDEs driven by both a Brownian motion and the compensated martingale of a random measure. Taoet al.[] discussed BSDEs with a sin-gular perturbed Markov chain; Taoet al.[, ] considered the regime-switching system modulated by a Markov chain based on Crepey and Matoussi [] and Crepey []. When the state space of a continuous-time Markov chain is described by the set of unit vectors inRd_{, Zhanget al.[] considered a system of Markov regime-switching model with}

Pois-son jumps. Different from the above works, based on Cohen and Elliott [], we discussed FBSDEs purely driven by a Markov chain rather than by a Brownian motion. Our model depends only on the state space of a Markov chain and leads to a special structure.

In Section , we formulate our problem; in Section , the preliminaries are given; in Sec-tion , we give our main result on the existence and uniqueness of the fully coupled FB-SDE on a Markov chain and present the proofs; in Section , we discuss the existence and uniqueness result of the FBSDE under other monotone assumptions; in the last section, we give an application of these theoretical results to stochastic optimal control problem through an example.

2 Formulation of the problem

Consider a continuous-time finite-state Markov chainm={mt,t∈[,T]}. Following the

convention of Elliottet al.[], we assume that it takes values in unit vectorsei inRd, wheredis the number of states of the chain.

the following representation:

mt=m+ t



Asmsds+Mt,

whereMtis a martingale (see []).

We consider the following forward-backward stochastic differential equations:

Xt=x+_tb(s,Xs,Ys,Zs)ds+_tσ(s–,Xs–,Ys–,Zs–)dMs, Yt=(XT) +

T

t f(s,Xs,Ys,Zs)ds– T

t Zs–dMs,

()

whereX,Y,Ztake values inRn,Rm,Rm×d,T>  is an arbitrary fixed number, andb,σ,

f,are functions of appropriate dimensions:

b:×[,T]×Rn×Rm×Rm×d−→Rn,

σ:×[,T]×Rn×Rm×Rm×d−→Rn×d,

f :×[,T]×Rn×Rm×Rm×d−→Rm,

:×Rn−→Rm.

We shall seek anFt-adapted triple (X,Y,Z) that satisfies the forward-backward stochas-tic differential equation () on [,T]P-almost surely. That is, our aim is to find anFt -adapted solution of ().

Noting that only thes-left limitσ(ω,s–,Xs–,Ys–,Zs–) enters into FBSDE (), without loss

of generality, we assume thatσ(s,Xs,Ys,Zs) is left-continuous insfor allwandX,Y. Since

Mis a semimartingale,X,Yiscàdlàgand adapted. We suppose the existence of the left limits ofZ. SoZ must have at most a countable number of discontinuities, and then it must be left-continuous at eachtexcept possibly on adt-null set. Hence, ifZs satisfies FBSDE (), then so doesZs–:

Xt=x+_tb(s,Xs,Ys,Zs–)ds+ t

σ(s–,Xs–,Ys–,Zs–)dMs, Yt=(XT) +_tTf(s,Xs,Ys,Zs–)ds–

T

t Zs–dMs.

SettingZ∗∗_t :=Zt–, we have a left-continuous processZ∗∗, which also satisfies the desired

equations. Therefore, writing the left limitsZt–is unnecessary since we simply assume that

our solution is left-continuous.

Based on these arguments, we rewrite FBSDE () as

Xt=x+_tb(s,Xs,Ys,Zs)ds+_tσ(s,Xs–,Ys–,Zs)dMs, Yt=(XT) +_tTf(s,Xs,Ys,Zs)ds–_tTZsdMs.

Recall that FBSDE () is equivalent to

Xt=x+_tb∗∗(s,Xs,Ys,Zs)ds+_tσ(s,Xs–,Ys–,Zs)dms,

Yt=(XT) +_tTf∗∗(s,Xs,Ys,Zs)ds–_tTZsdms, ()

where b∗∗(s,Xs,Ys,Zs) =b(s,Xs,Ys,Zs) –σ(s,Xs,Ys,Zs)Asms andf∗∗(s,Xs,Ys,Zs) =f(s,Xs, Ys,Zs) +ZsAsms.

3 Preliminaries

3.1 Preliminary notation

Elliottet al.[] obtained the optional quadratic variation ofMtas the matrix process

[M,M]t= t

s=

MsM_s∗,

where ‘∗’ means transpose.

Observing thatAis the rate matrix of the Markov chainm, the predictable quadratic variation is

M,Mt=

t



diag(Asms) –diag(ms)A∗s–Asdiag(ms)

ds.

This can be seen through considering

diag(mt) =diag(m) + t



diag(Asms)ds+

t



diag(dMs)

and

diag(mt) =mtm∗_t

=mm∗+ t



msm∗_sA∗_sds+

t



ms–dM∗s+ t



Asmsm∗_sds

+

t



dMsm∗s–+ t

s=

MsM∗_s.

Equating these two, we get

[M,M]t=Lt+ t



diag(Asms) –diag(ms)A∗_s–Asdiag(ms)

ds,

whereLis some martingale. This in turn suggests that

M,Mt=

t



diag(Asms) –diag(ms)A∗s–Asdiag(ms)

ds.

We will use the following notation:

By (·,·) we denote the usual inner product inRn_orRm_{; we use the usual Euclidean norm}

inRnandRm; and forz∈Rm×d, we define|z|={tr(zz∗)}_. Forz_∈Rm×d_,z_∈Rm×d_,

z,z =trzz ∗ ,

and foru_{= (x}_,y_,z_)∈Rn_×Rm_×Rm×d_,u_{= (x}_,y_,z_)∈Rn_×Rm_×Rm×d_,

We are given anm×nfull-rank matrixG. Foru= (x,y,z)∈Rn_×Rm_×Rm×d_{, let}

F(t,u) =–G∗f(t,u),Gb(t,u),  , H(t,u) =, ,Gσ(t,u) ,

whereGσ= (Gσ· · ·Gσd).

Furthermore, we define

C,DV=trCdiag(Asv) –diag(v)A∗_s–Asdiag(v)

D∗ , C _v=C,Cv,

wherevis a basis vector inRd_.

3.2 BSDEs on Markov chains

We recall the existence and uniqueness results for a solution to the following BSDE on a Markov chain:

Yt=(XT) –

T

t

F(ω,s,Ys,Zs)ds– T

t

F(s,Ys–) +Zs

dMs

for functions F :×[,T]× Rn×Rn×d and F :×[,T]×Rn; these functions

are assumed to be progressively measurable, that is,F(·,s,Ys,Zs) andF(·,s,Ys) areFt

-measurable for allt∈[,T].

Cohen and Elliott [] gave the following result about martingale representation.

Lemma  Any Rd_{-valued martingale L defined on (,Ft,P)can be represented as a}

stochastic(in this case,Stieltjes)integral with respect to the martingale process M up to equalityP-a.s.This representation is unique up to a dM,Mt×P-null set.That is,

Lt=L+ t



ZsdMs,

where Zsis a predictableRn×d-valued matrix process.

Cohen and Elliott [] also obtained the existence and uniqueness of a solution to the above BSDE.

Theorem  Assume the Lipschitz continuity on the generators Fand F,that is,that there exists c∈Rsuch that,for all s∈[,T],

EF

s,Y_s,Z_s –F

s,Y_s,Z_s ≤cEY_s–Y_s+cEZ_s–Z_s_M

s,

EF

s,Y_s_– –F

s,Y_s_– _M

s≤c

_EY s–Ys



.

Under this Lipschitz condition,the above equation has at most one solution up to indistin-guishability for Y and equality dM,Mt×P-a.s.for Y.

4 Existence and uniqueness for FBSDE (1)

Definition  We denote byM_(,T;Rn_{) the set of allR}n_{-valuedFt-adapted processes}

such that

E T



v(s)ds< +∞.

Definition  A triple of processes (X,Y,Z) :×[,T]→Rn×Rm×Rm×dis called an adapted solution of the FBSDE () if (X,Y,Z)∈M_(,T;Rn_×Rm_×Rm×d_{) and it satisfies}

FBSDE ()P-almost surely.

The adaptedness of the solution enables us to rewrite FBSDE () in a differential form:

⎧ ⎪ ⎨ ⎪ ⎩

dXt=b(t,Xt,Yt,Zt)dt+σ(t,Xt–,Yt–,Zt)dMt,

–dYt=f(t,Xt,Yt,Zt)dt–ZtdMt,

X=x, YT=(XT).

Now we give the main assumptions of our paper.

Assumption  For eachu= (x,y,z)∈Rn_×Rm_×Rm×d_{,F(·,u),H(·,u)∈M}_(,T;Rn_×

Rm_×Rm×d_{), and for eachx∈R}n_,(x)∈L_(,FT;Rn_{). Moreover,}

(i) F(t,u)is uniformly Lipschitz with respect tou; (ii) H(t,u)is uniformly Lipschitz with respect tou; (iii) (x)is uniformly Lipschitz with respect tox.

Assumption  There exists constantsc,c,csuch that

Ft,u –Ft,u ,u–u≤–cG

x–x –c_G∗y–y +G∗z–z  ,

Ht,u –Ht,u ,u–u≤–cG

x–x –c_G∗y–y +G∗z–z  ,

∀u_=x_,y_,z _,u_=x_,y_,z _∈Rn_×Rm_×Rm×d_{, P-a.s., a.e.t∈R}+_,

and

x –x ,Gx–x ≥cG

x–x , ∀x∈Rn,∀x∈Rn,

wherec,c, andcare given positive constants.

Now we give the main result of our paper.

Theorem  Let Assumptionsandhold.Then there exists a unique adapted solution

(X,Y,Z)of FBSDE().

Proof of uniqueness LetU_{= (X}_,Y_,Z_{) andU}_{= (X}_,Y_,Z_{) be two adapted solutions}

of FBSDE (). We set

(Xˆ,Yˆ,Zˆ) =X–X,Y–Y,Z–Z ,

⎧ ⎪ ⎨ ⎪ ⎩ ˆ

b(t) =b(t,U

t) –b(t,Ut), ˆ

σ(t) =σ(t,U_t) –σ(t,U_t),

ˆ

f(t) =f(t,U

Then we have

dXtˆ =bˆ(t,Xt,Yt,Zt)dt+σˆ(t,Xt–,Yt–,Zt)dMt,

–dYtˆ =fˆ(t,Xt,Yt,Zt)dt–Ztˆ dMt.

From Assumption  it follows that

Esup_t∈[,T]| ˆXt| +E

sup_t∈[,T]| ˆYt| _{< +∞.}

Using the Itô product rule of semimartingales, we have

Sincec,c, andcare given positive constants, we have|GX|ˆ ≡,|G∗Yˆ|≡,|G∗Z|ˆ ≡

. SoXˆ

s≡ ˆXs,Yˆs≡ ˆYs,Zˆs≡ ˆZs,dM,Mt×P-a.s.

We now consider the following family of FBSDEs parameterized byl∈[, ]:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

dX_tl= [( –l)c_(–G∗Y_tl) +lb(t,U_tl) +φt]dt

+ [( –l)c_(–G∗Zl

t) +lσ(t,Utl) +ψt]dMt,

–dYl

t = [( –l)cGXlt+lf(t,Utl) +γt]dt–ZtldMt, Xl

=x, YTl =l(XTl) + ( –l)GXTl +ξ,

()

whereφ,ψ, andγ are given processes inM_{(,T) with values inR}n_,Rn×d_{, andR}m_,

re-spectively. Clearly, whenl= ,ξ≡, the existence of a solution of FBSDE () implies the existence of that of FBSDE (). Whenl=  in FBSDE (), it is

⎧ ⎪ ⎨ ⎪ ⎩

dX_t= [–c_G∗Y_t+φt]dt+ [–cG∗Zt+ψt]dMt,

–dY

t = [cGXt+γt]dt–Zt dMt, Xl

=x, YT=GXT+ξ.

()

For proving the existence part of the theorem, we first need the following two lemmas.

Lemma  The following equation has a unique solution:

⎧ ⎪ ⎨ ⎪ ⎩

dXt= [–c_G∗Yt+φt]dt+ [–cG∗Zt+ψt]dMt,

–dYt= [cGXt+γt]dt–ZtdMt, X=x, YT=λGXT+ξ,

()

whereλis a nonnegative constant.

Proof We note that the matrixGis of full rank. The proof of the existence for FBSDE () can be divided into two cases,n≤mandn>m.

For the first case, the matrixG∗Gis of full rank. Let

⎛ ⎜ ⎝

X Y Z

⎞ ⎟ ⎠=

⎛ ⎜ ⎝

X G∗Y G∗Z

⎞ ⎟ ⎠,

Y Z

=

(Im–G(G∗G)–_G∗_)Y

(Im–G(G∗G)–G∗)Z

.

MultiplyingG∗for (Y,Z) on both sides of the BSDE yields

⎧ ⎪ ⎨ ⎪ ⎩

dX_t= [–c_Y_t+φt]dt+ [–c_Z_t+ψt]dMt, –dY_t= [cG∗GXt+G∗γt]dt–ZtdMt, X_ =x, Y_T =λG∗GX_T +G∗ξ.

()

Similarly, multiplying (Im–G(G∗G)–_G∗_{) on both sides of the same equation gives}

–dY_t= (Im–G(G∗G)–_G∗_)γ

Obviously, the pair (Y,Z) is uniquely determined (see Cohen and Elliott []). The uniqueness of (X,Y,Z) follows from Theorem . In order to solve FBSDE (), we intro-duce the followingn×n-symmetric valued ODE, which is known as the matrix-Riccati equation:

–K˙(t) = –c_K_+c

G∗G, t∈[,T], KT=λG∗G.

It is well known that this equation has a unique nonnegative solutionK∈C_([,T);Sn_),

whereSnstands for the space of alln×n-symmetric matrices. We then study the solution (p,q)∈M_(,T;Rn+n×d_{) of the following linear simple BSDE:}

–dpt= [–cK(t)pt+K(t)φt+G∗γt]dt+ [K(t)ψt– (In+cK(t))qt]dMt, t∈[,T], pT=G∗ξ.

We now considerX_tas the solution of the SDE

dX_t= [–c_(K(t)X_t+pt) +φt]dt+ [ψt–cqt]dMt, X_=x.

Then we can easily check that (X_t,Y_t,Z_t) = (X_t,K(t)X_t+pt,qt) is a solution of FBSDE (). Once (X,Y,Z) and (Y,Z) are resolved, we can obtain the triple (X,Y,Z) uniquely by

⎛ ⎜ ⎝ X Y Z

⎞ ⎟ ⎠=

⎛ ⎜ ⎝

X G(G∗G)–_Y+Y G(G∗G)–_Z+Z ⎞ ⎟ ⎠.

For the second case, the matrixGG∗is of full rank. We set

⎛ ⎜ ⎜ ⎜ ⎝

X Y Z X

⎞ ⎟ ⎟ ⎟ ⎠=

⎛ ⎜ ⎜ ⎜ ⎝

GX Y Z

(In–G∗(GG∗)–_G)X ⎞ ⎟ ⎟ ⎟ ⎠,

whereXis the unique solution of the following linear SDE:

dX_t= (In–G∗(GG∗)–G)φtdt+ (In–G∗(GG∗)–G)ϕtdMt, X= (In–G∗(GG∗)–G)x.

The triple (X,Y,Z) solves the FBSDE

⎧ ⎪ ⎨ ⎪ ⎩

dX_t= [–c_GG∗Y_t+Gφt]dt+ [–cGG∗Zt+Gψt]dMt,

–dY_t= [cXt+γt]dt–ZtdMt, X_ =Gx, Y_T =λX_T+ξ.

In order to solve this equation, we introduce the followingm×m-symmetric matrix-valued matrix-Raccati equation:

–K˙(t) =cIm–cKGG∗K, t∈[,T], K(T) =λIm.

It is well known that this equation has a unique nonnegative solutionK∈C_([,T);Sm_),

where Sm _{stands for the space of allm×m-symmetric matrices. We then look for the}

solution (p,q)∈M_(,T;Rm+m×d_{) of the following linear simple BSDE:} ⎧

⎪ ⎨ ⎪ ⎩

–dpt= [–c_K(t)GG∗pt+K(t)Gφt+γt]dt+ (K(t)Gψt– (Im+c_K(t)GG∗)qt)dMt,

t∈[,T],

pT=ξ.

We now letX_tbe the solution of the following SDE:

dX_t= [–c_GG∗(K(t)X_t+γt) +Gφt]dt+ [Gψt–cGG∗qt]dMt, X_=Gx.

Then we can easily check that (X_t,Y_t,Z_t) = (X_t,K(t)X_t+pt,qt) is a solution of FBSDE (). Once (X,X,Y,Z) are resolved, then by the definition we can obtain the triple (X,Y,Z) uniquely as

⎛ ⎜ ⎝ X Y Z

⎞ ⎟ ⎠=

⎛ ⎜ ⎝

G∗(GG∗)–_X+X Y Z

⎞ ⎟ ⎠.

The proof is complete.

Lemma  We assume Assumptionsand.If for an l∈[, ),there exists a solution

(Xl_,Yl_,Zl_{)of FBSDE(),then there exists a positive constantδ}

such that for eachδ∈

[,δ],there exists a solution(Xl+δ,Yl+δ,Zl+δ)of FBSDE()for l=l+δ.

Proof Since for all φ ∈M_(,T;Rn_{), γ ∈M}_(,T;Rm_{), ψ ∈M}_(,T;Rn×d_{), ξ ∈L}_(, FT,P),x∈Rn_,l

∈[, ), there exists a unique solution of (), for everyUs= (Xs,Ys,Zs)∈ M(,T;Rn+m+m×d), there exists a unique tripleus= (xs,ys,zs)∈M(,T;Rn+m+m×d) satis-fying the following FBSDE:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

dxt= [( –l)c(–G∗yt) +lb(t,ut) +δb(t,Ut) +δc(G∗Yt) +φt]dt

+ [( –l)c(–G∗zt) +lσ(t,ut) +δσ(t,Ut) +δc(G∗Zt) +ψt]dMt,

–dyt= [( –l)cGxt+lf(t,ut) +δ(–cGXt) +δf(t,Ut) +γt]dt–ztdMt, x=x, yT=l(xT) + ( –l)GxT+δ((XT) –GXT) +ξ.

We want to prove that the mapping defined by

Il+δ(U×XT) =u×xT:M

_,T;Rn+m+m×d _×L_(,FT,P)

→M,T;Rn+m+m×d ×L(,FT,P)

Let U×X_T = (X,Y,Z)×X_T ∈M_(,T;Rn+m+m×d_)×L_{(,FT,P) and u×x}

Using the Itô product rule of semimartingales for the product (Gˆxs,ysˆ), taking expecta-tions, and evaluating att=T yield

For the difference of the solutions (yˆ,ˆz) = (y–y,z–z), we apply the usual technique to the BSDE and derive that

E

Similarly, for the difference of the solutionxˆ=x–x, we apply the usual technique to the forward part:

Combing the above estimates, we get

E

such that, for eachδ∈[,δ], FBSDEs () forl=l+δhas a unique solution. We can repeat

this process forNtimes with ≤Nδ<  +δ. It then follows that, in particular, FBSDE

() forl=  withξ≡ has a unique solution. The proof is complete.

5 Another existence and uniqueness theorem

Next, we give another existence and uniqueness theorem. First, we give another assump-tion.

Assumption 

There exist constantsc,c,csuch that

Ft,u –Ft,u ,u–u≥cG

x–x +c_G∗y–y +G∗z–z  ,

Ht,u –Ht,u ,u–u≥cG

x–x +c_G∗y–y +G∗z–z  ,

∀u=x,y,z ,u=x,y,z ∈Rn×Rm×Rm×d, P-a.s., a.e.t∈R+,

and

x –x ,Gx–x ≤–cG

x–x , ∀x∈Rn,∀x∈Rn,

wherec,c, andcare given positive constants.

Theorem  Let Assumptionsandhold.Then there exists a unique adapted solution

(X,Y,Z)for FBSDE().

Proof of uniqueness Using the same procedure as in the proof of uniqueness of Theorem , by Assumption  we get

E(YTˆ ,GXTˆ )

=EX_T –X_T ,GX_T –X_T

=E T



Fs,U –Fs,U ,U–Uds

+E T



Hs,U –Hs,U ,U–UdM,Ms

≥cE T



|GX|ˆ _ds+c E

T



G∗Yˆ+G∗Zˆ ds

+cE T



|GX|ˆ _dM,Ms+c E

T



_G∗_Yˆ

+G∗Zˆ dM,Ms.

Thus, we deduce that

c

E T



|GXˆ|ds+E T



|GXˆ|dM,Ms

+c_

E T



_G∗_Yˆ

+G∗Zˆ ds+E T



_G∗_Yˆ

+G∗Zˆ dM,Ms

+c|GX|ˆ 

Sincec,c, andcare given positive constants, we have|GX|ˆ ≡,|G∗Yˆ|≡,|G∗Z|ˆ ≡

. SoXˆ

s≡ ˆXs,Yˆs≡ ˆYs,Zˆs≡ ˆZs,dM,Mt×P-a.s.

We now consider the following family of FBSDEs parameterized byl∈[, ]:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

dXl

t= [( –l)c(G∗Ytl) +lb(t,Utl) +φt]dt

+ [lσ(t,U_tl) + ( –l)c_(G∗Zl_t) +ψt]dMt,

–dYl

t = [–( –l)cGXtl+lf(t,Utl) +γt]dt–ZtldMt, Xl

=x, YTl =l(XTl) + ( –l)GXTl +ξ,

()

whereφ,ψ, andγ are given processes inM_{(,T) with values inR}n_,Rn×d_{, andR}m_,

respec-tively. Clearly, whenl=  andξ≡, the existence of a solution of FBSDE () implies that of FBSDE (). Whenl=  in FBSDEs (), it is

⎧ ⎪ ⎨ ⎪ ⎩

dX

t = [cG∗Yt+φt]dt+ [cG∗Zt+ψt]dMt,

–dY_t= [–cGXt+γt]dt–ZtdMt, X_l =x, Y_T=GX_T+ξ.

()

For proving the existence part of the theorem, we first need the following two lemmas.

Lemma  The following equation has a unique solution:

⎧ ⎪ ⎨ ⎪ ⎩

dXt= [c_G∗Yt+φt]dt+ [cG∗Zt+ψt]dMt,

–dYt= [–cGXt+γt]dt–ZtdMt, X=x, YT=λGXT+ξ,

()

whereλis a nonnegative constant.

Lemma  Let Assumptions  and  hold. If for an l ∈[, ), there exists a solution

(Xl_,Yl_,Zl_{) of FBSDE (), then there exists a positive constant δ}

 such that for each δ∈[,δ],there exists a solution(Xl+δ,Yl+δ,Zl+δ)of FBSDE()for l=l+δ.

Remark  The proofs of Lemma  and Lemma  are similar to those of Lemma  and

Lemma .

Proof of existence From Lemma  we immediately see that whenλ=  in FBSDE (), FBSDE () forl=  (that is, FBSDE ()) has a unique solution. It then follows from Lemma  that there exists a positive constantδdepending on the Lipschitz constants,c,c,c,

andTsuch that, for eachδ∈[,δ], FBSDE () forl=l+δhas a unique solution. We can

repeat this process forN times with ≤Nδ<  +δ. It then follows that, in particular,

FBSDEs () forl=  withξ≡ has a unique solution.

The proof is complete.

6 Application to stochastic optimal control problems

the process with the unit vectorseiinRk_{, wherekis the number of states of the chain. The}

rate matrix ofmtis denoted byAt.

All the stochastic processes are defined on the filtered probability space (,F,Ft,P), where{Ft}is the completed natural filtration generated by theσ-fieldsσ({ms,s≤t},F∈ FT:P(F) = ), andF=FT. As noted before, the martingaleMt=mt–m–

t

Asmsds

is used to drive the controlled state processXt. Next, we define the function: [,T]×

{ei|i= , , . . . ,k} →Sk_by

(s,ei) :=diag(Asei) –diag(ei)A∗_s–Asdiag(ei).

Note that

M,Mt=

t



diag(Asms) –diag(ms)A∗_s–Asdiag(ms)

ds

=

t



(s,ms)ds.

Consider the one-dimensional linear quadratic optimal control problem

dXt= [AXt+But]dt+ [CXt–+Dut]dMt, X=x,

()

and the cost function

Ju(·) =E

 

T



QX_t+Ru_t dt+ Gx

 T

, ()

where the parametersA,B,Q,R, Gare positive constant numbers, andC,DareR×k vectors.

Denote U ={u: [,T]× → R|uis{Ft}-predictable,E_T|u(s)|ds < +∞}. The stochastic control problem is to find an optimal controlu∗∈Usuch that

Ju∗ =inf u∈UJ(u).

In order to solve the problem, we define the HamiltonianH: [,T]×R×R×R×R×k_× {ei|i= , , . . . ,k} →Rby

H(t,x,u,p,s,ei) =  Qx

₊

Ru

_{+ (Ax+Bu)p+ (Cx+Du)(t,ei)s}∗_.

The adjoint equation is given by the following BSDE:

⎧ ⎪ ⎨ ⎪ ⎩

dpt= –Hx(t,Xt,ut,pt,st,mt)dt+stdMt

= –(QXt+Apt+Ct(t,mt)s∗_t)dt+stdMt,

pT=GXT.

()

Theorem  Given the assumptions on the parameters above,the coupled FBSDE solvable with the optimal controlled state processXtˆ and the optimal control defined by

ˆ

ut= –R–Bˆpt–+D(t,mt)ˆs∗t . ()

Proof First, it is easy to check that FBSDE () satisfies Assumptions  and . So, according to Theorem , there exists a unique adapted solution (Xtˆ ,ptˆ ,ˆst)∈M_{(,T;R×R×R}×k_).

So, according to (),uˆ∈U, andXˆ is the corresponding state process. Next, for any fixed

u∈Uwith the corresponding state processXt, consider

for some local martingaleL(see []). So

By the definition ofHwe have

E

Adding the two equations, we have

J(u) –J(uˆ)≥E

By the convexity of  Qx

Remark  Differently from the Brownian motion case, the optimal control and the Hamil-tonian additionally depend on the state of the Markov chain. This is because Markov chains are discontinuous stochastic processes with finite variation and the quadratic processes of the martingales generated by Markov chains are quite different from the continuous-time diffusions.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All authors jointly worked on the results, and they read and approved the final manuscript.

Author details

1_{Qilu Institute of Finance, Shandong University, Jinan, 250100, P.R. China.}2_{School of Mathematical Sciences, Shandong}

Normal University, Jinan, 250014, P.R. China.

Acknowledgements

This research are supported by School of Mathematical Sciences project (shxzhxxm201503) of Shandong Normal University, National Natural Science Foundation project (11301309) of China, National Natural Science Foundation project (11201271) of China, and Shandong Provincial Scientific Research Foundation for Excellent Young Scientists

(BS2013SF003). We would like to thank the reviewers, whose careful reading, help suggestions, and valuable comments helped us to improve the manuscript.

Received: 11 May 2015 Accepted: 10 May 2016 References

1. Pardoux, E, Peng, S: Adapted solution of a backward stochastic differential equations. Syst. Control Lett.14, 55-61 (1990)

2. Tang, S, Li, X: Necessary conditions for optimal control of stochastic systems with random jumps. SIAM J. Control Optim.32(5), 1447-1475 (1994)

3. Nualart, D, Schoutens, W: Backward stochastic differential equations and Feynman-Kac formula for Levy processes, with applications in finance. Bernoulli7(5), 761-776 (2001)

4. Cohen, S, Elliott, R: Solutions of backward stochastic differential equations on Markov chains. Commun. Stoch. Anal.

2(2), 251-262 (2008)

5. Cohen, S, Elliott, R: Comparisons for backward stochastic differential equations on Markov chains and related no-arbitrage conditions. Ann. Appl. Probab.20(1), 267-311 (2010)

6. Cohen, S, Hu, Y: Ergodic BSDEs driven by Markov chains. SIAM J. Control Optim.51(5), 4138-4168 (2013) 7. Ma, J, Yong, J: Forward-Backward Stochastic Differential Equations and Their Applications. Lecture Notes in

Mathematics. Springer, Berlin (1999)

8. Buchdahn, R, Hu, Y: Hedging contingent claims for a large investor in an incomplete market. Adv. Appl. Probab.30(1), 239-255 (1998)

9. Hu, Y: Potential kernels associated with a filtration and forward-backward SDEs. Potential Anal.10(2), 103-118 (1999) 10. Antonelli, F: Backward-forward stochastic differential equations. Ann. Appl. Probab.3, 777-793 (1993)

11. Pardoux, E, Tang, S: Forward-backward stochastic differential equations and quasilinear parabolic PDEs. Probab. Theory Relat. Fields114(2), 123-150 (1999)

12. Ma, J, Protter, P, Yong, J: Solving forward-backward stochastic differential equations explicitly: a four step scheme. Probab. Theory Relat. Fields98(3), 339-359 (1994)

13. Hu, Y, Peng, S: Solutions of forward-backward stochastic differential equations. Probab. Theory Relat. Fields103(2), 273-283 (1995)

14. Peng, S, Wu, Z: Fully coupled forward-backward stochastic differential equations. SIAM J. Control Optim.37(3), 825-843 (1999)

15. Yong, J: Forward-backward stochastic differential equations with mixed initial-terminal conditions. Trans. Am. Math. Soc.362(2), 1047-1096 (2010)

16. Ma, J, Wu, Z, Zhang, D, Zhang, J: On wellposedness of forward-backward SDEs: a unified approach. Ann. Appl. Probab.

25(4), 2168-2214 (2015)

17. Briand, P, Hu, Y: Probabilistic approach to singular perturbations of semilinear and quasilinear parabolic PDEs. Nonlinear Anal., Theory Methods Appl.35(7), 815-831 (1999)

18. Zhang, X, Elliott, R, Siu, T: A stochastic maximum principle for a Markov regime-switching jump-diffusion model and its application to finance. SIAM J. Control Optim.50, 964-990 (2012)

19. Crepey, S, Matoussi, A: Reflected and doubly reflected BSDEs with jumps: a priori estimates and comparison. Ann. Appl. Probab.18, 2041-2069 (2008)

20. Crepey, S: About the Pricing Equations in Finance. Paris-Princeton Lectures on Mathematical Finance. Springer, Berlin (2010)

21. Tao, R, Wu, Z, Zhang, Q: BSDEs with regime switching: weak convergence and applications. J. Math. Anal. Appl.407, 97-111 (2013)

22. Tao, R, Wu, Z: Maximum principle for optimal control problems of forward-backward regime-switching system and applications. Syst. Control Lett.61, 911-917 (2012)

23. Tao, R, Wu, Z, Zhang, Q: Optimal switching under a regime-switching model with two-time-scale Markov chains. Multiscale Model. Simul.13, 99-131 (2015)