Relationship between Maximum Principle and Dynamic Programming in Stochastic Differential Games and Applications

(1)

http://dx.doi.org/10.4236/ajor.2013.36043

Relationship between Maximum Principle and Dynamic

Programming in Stochastic Differential

Games and Applications

Jingtao Shi

School of Mathematics, Shandong University, Jinan, China Email: [email protected]

Received July 22, 2013; revised August 22, 2013; accepted August 30,2013

Copyright © 2013 Jingtao Shi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

ABSTRACT

This paper is concerned with the relationship between maximum principle and dynamic programming in zero-sum stochastic differential games. Under the assumption that the value function is smooth enough, relations among the adjoint processes, the generalized Hamiltonian function and the value function are given. A portfolio optimization problem under model uncertainty in the financial market is discussed to show the applications of our result.

Keywords: Stochastic Optimal Control; Stochastic Differential Games; Dynamic Programming; Maximum Principle;

Portfolio Optimization; Model Uncertainty

1. Introduction

Game theory has been an active area of research and a useful tool in many applications, particularly in biology and economic. Among others, there are two main approaches to study differential game problems. One approach is Bellman’s dynamic programming, which re- lates the saddle points or Nash equilibrium points to some partial differential Equations (PDEs) which are known as the Hamilton-Jacobi-Bellman-Isaacs (HJBI) Equations (see Elliott [1], Fleming and Souganidis [2], Buckdahn et al. [3], Mataramvura and Oksendal [4]). The other approach is Pontryagin’s maximum principle, which finds solutions to the differential games via some Hamiltonian function and adjoint processes (see Tang and Li [5], An and Oksendal [6]).

Hence, a natural question arises: Are there any relations between these two methods? For stochastic control problems, such a topic has been discussed by many au- thors (see Bensoussan [7], Zhou [8], Yong and Zhou [9], Framstad et al. [10], Shi and Wu [11], Donnelly [12], etc.) However, to the best of our knowledge, the study on the relationship between maximum principle and dynamic programming for stochastic differential games is quite lacking in literature.

In this paper, we consider one kind of zero-sum stochastic differential game problem within the frame work

of Mataramvura and Oksendal [4] and An and Oksendal [6]. However, we don’t consider jumps. This more gen- eral case will appear in our forthcoming paper. For our problem in this paper, [4] related its saddle point to some HJBI Equation and obtained the stochastic verification theorem. [6] proves both sufficient and necessary maximum principles, which state some conditions of optimality via the Hamiltonian function and adjoint Equation. The main contribution of this paper is that we connect the maximum principle of [6] with the dynamic programming of [4], and obtain relations among the adjoint processes, the generalized Hamiltonian function and the value function under the assumption that the value function is enough smooth. As applications, we discuss a portfolio optimization problem under model uncertainty in the financial market. In this problem, the optimal portfolio strategies for the trader (representative agent) and the “worst case scenarios” (see Peskir and Shorish [13], Korn and Menkens [14]) for the market, derived from both maximum principle and dynamic programming approaches independently, coincide. The relation that we obtained in our main result is illustrated.

(2)

and Hamiltonian function, and the stochastic verification theorem [4] by HJBI Equation. In Section 3, we prove the relationship between maximum principle and dynamic programming for our zero-sum stochastic differential game problem, under the assumption that the value function is smooth enough. A portfolio optimization problem under model uncertainty in the financial market is discussed in Section 4, to show the applications of our result.

Notations: throughout this paper, we denote by _Rn

the space of n-dimensional Euclidean space, by _Rn d

the space of n d matrices, by _Sn_{the space of}_{n n}_

symmetric matrices. , and |.| denote the scalar product and norm in the Euclidean space, respectively. 

ap-pearing in the superscripts denotes the transpose of a matrix.

2. Problem Statement and Preliminaries

Let T 0 be given, suppose that the dynamics of a stochastic system is described by a stochastic differential Equation (SDE) on a complete probability space



, , P



of the form

 



   



   





 

, ; , ;

d , , d

, , d ,

,

t x u t x u

t x u

X s b s X s u s s

s X s u s W s

X t x



 _



 _

 

 

(1)

with initial time t



0,T



and initial state _{x R}_ n_.

Here



W t

 



₀_{ }_{t T} is a d-dimensional standard Brown- ian motion. For given t



0,T



, we suppose the filtra- tion

 

st _{t s}_{ }_T is generated as the following

 





s _t

t

r s

W r W t _{ }

  

  ,

where  contains all P-null sets in  and ₁₂

denotes the -field generated by ₁₂. In particular, if the initial time t0, we write _st

s 

  . In the above,

 

: 0, n n

b T R  U R ,

 

: 0,_T _Rn _U _Rn d

 _ _{ } 

are given continuous functions, where _U__Rk_{is non-}

empty and convex. The U-valued process u t

 

u t

 

, ,

  is the control process.

For any t



0,T



, we denote by U t T

 

, the set of

 

s _t t

s

 -adapted processes. For given u

 

. U t T

 

,

and _{x R}_ n_{, an}_Rn_{-valued process}_Xt x u, ;

 

_. _{is called} a solution to (1) if it is an t

s

 -adapted process such that (1) holds. We refer to such u

 

. U t T

 

, as an admissible control and



_Xt x u, ;

   

_{. , .}_u



_{as an admissible} pair.

We make the following assumption.

(H1) There exists a constant C0 such that for all

 

0, , ,ˆ n, ,ˆ

s T x x R u u U  , we have



, ,

 

, ,ˆ ˆ



ˆ ˆ b s x u b s x u    x x u u ,



s x u, ,





s x u, ,ˆ ˆ



x xˆ u uˆ

      ,



, ,





, ,





1



b s x u  s x u   x .

For any u

 

. U t T

 

, , under (H1), it is obvious that SDE (1) has a unique solution _Xt x u, ;

 

_. _.

Let _f : 0,

 

_T __Rn_{ }_U _R_and_{g R}_: n __R_{be con-}

tinuous functions. For any

 

_{t x}, _

 

0,_T __Rn_{and ad-}

missible control u

 

. U t T

 

, , we define the following performance functional







   



 





, ; , ;

, ; , , d

.

T _{t x u}

t

t x u

J t x u E f s X s u s s

g X T

  



 _



(2)

Now suppose that the control process has the form

 

.



   

. ,π .



u   , (3)

where  and π are valued in two sets K1 and K2, respectively. We let 

 

t T, and 

 

t T, be given fami- lies of admissible controls 

 

_{s t s T}_{ } and π

 

π_{s t s T}_{ } , respectively. The zero-sum stochastic differential game problem is to find



_*_,_π*



_{ }

   

_{t T}_, _{ } _{t T}_, _{such that}

 





   





  _{ }





* *

, ,

, _,

, , ; ,π

sup inf , ; ,π

inf sup , ; ,π ,

t T t T

t T _{t T}

V t x J t x

J t x

J t x 



 _



 

 _



 

 _ _

 

 

 _ _

 

(4)

for given

 

_{t x}, _

 

0,_T __Rn_{, with}_{V T x}



_,



__{g x x R}

 

_, _ n_.

Such a control process (pair)



_*_,_π*



_{is called an opti-} mal control or a saddle point of our zero-sum stochastic differential game problem (if it exists). And the corresponding solution _X*

 

_. __Xt x u, ;

 

_. _{to (1) is called the} optimal state.

The intuitive idea is that there are two players, I and II. Player I controls  and Player II controls . The ac- tions of the two players are antagonistic, which means that between Players I and II there is a payoff J t x



, ; , π



which is a cost for player I and a reward for Player II. We now define the Hamiltonian function

 

1 2

: 0, n n n d

H T R K K R R R

by





















, , ,π, , : , , ,π ,

, , ,π , , ,π .

H s x p q b s x p

tr s x q f s x

 

  



(3)

In addition, we need the following assumption. (H2) B, σ, f are continuously differentiable in



x, , π



and g is continuously differentiable in x. Moreover,

,

x x

b  are bounded and there exists a constant C0

such that for all s

 

0,T ,



,π



K₁K₂, we have



, , ,π



 



1



, n.

x x

f s x  g x C  x  x R

The adjoint Equation in the unknown s-adapted pro-

cesses _{p R q R}_ n, _ n d _{is the backward stochastic dif-}

ferential Equation (BSDE) of the form

 



         



 



 



, ; ,π

d , , ,π , ,

d ,

.

t x

t x x

H

p s s X s s s p s q s

s q s W s

p T g X T



 

 _{ }

 _

 _  

  

(6)

For any



,π



 

   

0,T   0,T , under (H2), we know that BSDE (6) admits a unique _s-adapted solution



p

   

. . .q



.

We now can state the following sufficient maximum principle which is Corollary 2.1 in An and Oksendal [6].

Lemma2.1 Let (H1), (H2) hold and x R n be fixed.

Suppose that



,π



 

   

0,T   0,T

with corresponding state process _X_ˆ0,x

 

_. __X_ˆ0, ; ,πxˆ ˆ

 

_. _.

Let _X0, ;πx

 

_. _and_X0, ;x

 

_. _be _{0, ; ,π}ˆ

_{ }

.

x

X  and

 

ˆ

 

0, ;x _. 0, ; ,πx _.

X   X  , respectively. Suppose that there exists a solution



pˆ

   

. , .qˆ



to the corresponding adjoint Equation (6). Moreover, suppose that for all

 

0,

s T , the following minimum/maximum conditions hold:

 

     





         





   





1

2

0, ;

0,

0, ;π π

ˆ

inf , , ,π , ,

ˆ

ˆ _ˆ

, , ,π , ,

ˆ

sup , , ,π, , ,

. .

x K

x

x K

E H s X s s p s q s

E H s X s s s p s q s

E H s X s s p s q s

P a s



 





 

 

 

  

 

 _ _



(7)

1) Suppose that for all s

 

0,T g x,

 

is concave and

 

_x_,_π __{H s x}



_{, ,}_ˆ

 

_s _,_π_,_{p s q s}_ˆ

   

_,_ˆ



is concave. Then



_{0, ; ,}ˆ _π_ˆ

 

_{0, ; ,}ˆ_π



_, _π

J x J x   ,

and



_{0, ; ,}ˆ _π_ˆ



_sup_π



_{0, ; ,}ˆ _π



J x J x



 .

2) Suppose that for all s

 

0,T g x,

 

is convex and

 

x, H s x



, , , πˆ

     

s p s q s,ˆ ,ˆ



is convex. Then



0, ; ,ˆ ˆπ





0, ; ,πˆ



,

J x J x    ,

and



0, ; ,ˆ ˆπ



inf



0, ; ,πˆ



J x J x



 



 .

3) If both Cases (1) and (2) hold (which implies, in particular, that gis an affine function), then



_*_,_π*



_

 

_ˆ ˆ_,_π

is an optimal control (saddle point) and

 









π

sup inf 0, ; ,π

inf sup 0, ; ,π .

V x J x

J x



 _



 



 _



 

 _ _

 

(8)

Next, when the control process u



,π



   is Markovian, then we can define the generator _A,π_of diffusion system (1) by

 

   





 



   



 

,π

2 2

, ,

, , ,π , ,

1

, , ,π , ,

2

A t x t x

t

b t x x x t x

x

tr t x x x t x

x 



 

 



 

 



 



  

 _ _



 

(9)

where _

 

_.,. __C1,2



 

_0,_T __{R R}n_;



_.

The following result is a stochastic verification theorem of optimality, which is an immediate corollary of Theorem 3.2 in Mataramvura and Oksendal [4].

Lemma2.2 Let (H1), (H2) hold and

 

t x, 

 

0,T Rn

be fixed. Suppose that there exists a

 

_.,. 1,2



 

_0, n_;



V C T R R

and a Markovian control process



ˆ

   

x ,πˆ x



  

such that

1) _{A V t x},πˆ

 

_, __{f t x}



_{, , ,}_ _π_ˆ

 

_x



_{    }_0, _ _,_{x R}n_,

2) _{A V t x}ˆ,π

 

, _ _{f t x}



, ,_ˆ

 

_x ,π



_{    }0, π ,_{x R}n,

3) _{A V t x}ˆ ˆ,π

 

_, __{f t x}



_{, ,}_ˆ

   

_x _,_π_ˆ _x



_{  }_0, _{x R}n_,

4) 



,π



  ,

 



0, ; ,π





0, ; ,π

 



lim , x x

t T t X t h X T

 





  ,

5) the family



0, ; ,x π

 



K

X 



 is uniformly integrable

for all _{x R}_ n_,



__,_π



_{  }_{, where}_K _{is the set of}

stopping times

T

(4)

 





















π π

π

sup inf 0, ; ,π inf sup 0, ; ,π

ˆ _ˆ

sup 0, ; ,π inf 0, ; ,π

ˆ ˆ 0, ; ,π ,

V x J x J x

J x J x

J x

 



 



 

 

 

 

  _ _

 

 



(11)

and



_*_,_π*



_

 

__{ˆ ˆ}_,_π _{is an optimal Markovian control.}

3. Main Result

In this section, we investigate the relationship between maximum principle and dynamic programming for our zero-sum stochastic differential game problem. The main contribution is that we find the connection between the value function V , the adjoint processes p q, and the following generalized Hamiltonian function

 

1 2

: 0, n n n

G T R K K R S R

defined by





















, , ,π, , : , , ,π , 1

, , ,π , , ,π .

2

G t x p A b t x p

tr t x A f t x

 

  



   (12)

Our main result is the following.

Theorem3.1 Let (H1), (H2) hold and

 

t x, 

 

0,T Rn

be fixed. Suppose that



_*_,_π*



_{is an optimal Mark-} ovian control, and _X*

 

_. _{is the corresponding optimal} state. Suppose that the value function

 

_.,. 1,2



 

_0, n_;



V C T R R ,

then (1)

 





 







 



*

* *

, ; ,π

, ; ,π *

2

, ; ,π , ; ,π 2

,

, , ,π ,

, , , ,

t x

t x t x

V

s X s

s

G s X s s

V _{s X} _s V _{s X} _s

x x



 

 

 

   



 



  _

(13)

 

,s t T P a s, , . .,



    

(2)

 





   

 







 



*

* *

, ; ,π

, ; ,π *

2

, ; ,π , ; ,π 2

,

, , ,

π, , , , ,

t x

t x t x

V _{s X} _s

s

G s X s s

V V

s X s s X s

x x



 

 

 

   



 



  _

(14)

 

π ,s t T P a s, , . .,

     and

(3)

 





     

 







 



* * * *

2

* *

2

, , , ,π ,

, , , ,

V _{s X s} _{G s X s} _s _s

s

V V

s X s s X s

x x



 

 _{ }

 _



 



  _

(15)

 

, , . .

s t T P a s

  

Further, suppose that

 

_.,. 1,3



 

_0, n_;



V C T R R

and Vsx is also continuous. For any s

 

t T, , define

 



 



 



 



     





* *

, ; ,π 2

, ; ,π * *

, ,

,

, , ,π ,

t x

V

p s s X s

x V

q s s X s

x

s X s s s



 



 _

 _





 _

 _



 _

 

(16)

then



p

   

. , .q



solves the adjoint Equation (6).

Proof. (13), (15) can be obtained from the HJBI Equa-

tion (10), by the definitions of the generator _A,π_{in (9)} and the generalized Hamiltonian function G in (12).

We proceed to prove the second part. If

 

_.,. 1,3



 

_0, n_;



V C T R R

and V_sx is also continuous, then from (15), we have

 

   

 

*

2

* *

2

, , , ,π , , , , 0.

x X s

V _{s x G s x} _s _s V _{s x} V _{s x}

x s  x x _



 

  _    _

  

     

This is equivalent to

 







 





     



     







 









     





 



     







 





     





     



2 2

* * * * *

2

3

* * * * * * * *

3 2

* * * * * * * * * *

2

, , , , ,π

1

, , ,π , tr , , ,π ,

2

tr , , ,π , , , ,π , , ,π 0,

V V

s X s s X s b s X s s s

s x x

b _{s X s} _s _s V _{s X s} _{s X s} _s _s V _{s X s}

x x x

V f

s X s s s s X s s X s s s s X s s s

x x x



  

 _ _ _ _

 



  

 

  

   

  _  _



 

     

 __ _ _ __ 

  

  

  

 _

(5)

where

 

33

 

22 1

 

22

tr : tr , , tr ,

n

V V V

x x

x x x

     

   __         _  _  _  _ _   _ _  _

   

   

      





  

with

1

, , .

n

V V V

x x x

__ _ __ _  _  _ _ _ _  __ _ __ _  _

  

 _{On the other hand, applying Ito’s formula to}

 



_, *



V

s X s x



 , we get

 







 





 





     









     





 



 







     



 

     







 





     



     





2 2

* * * * * *

2 3

* * * *

3 2

* * * *

2

* * * * * * *

* * *

d , , , , , ,π

1_tr _, _, _,_π _, _d

2

, , , ,π d

, , ,π , , , ,π

tr , , ,π

V _{s X s} V _{s X s} V _{s X s b s X s} _s _s

x s x x

V

s X s s s s X s s

x V

s X s s X s s s W s

x

b V f

s X s s s s X s s X s s s

x x x

s X s s s

x



 

 

 

 _



  

 

 _  



  

  

 

 

 



  



 _ 

  

 

 

 _{ }

 



 







     



 







     



 

2

* * * *

2 2

* * * *

2

, , , ,π d

, , , ,π d .

V

s X s s X s s s s

x

V

s X s s X s s s W s

x

 



 _ 

 

  

  

 _

 

 

 

 





Note that

 



_, *





*

 



x

V _{T X T} _{g X T}

x

 _

 .

Hence, by the uniqueness of the solutions to (6), we obtain (16). The proof is complete.□

4. Applications

In this section, we will discuss a portfolio optimization problem under model uncertainty in the financial market, where the problem is put into the framework of a zero-sum stochastic differential game. The optimal portfolio strategies for the investor and the “worst case scenarios” for the market, derived both from maximum principle and dynamic programming approaches independently, coincide. The relation that we obtained in our main result Theorem 3.1 is illustrated.

Suppose that the investors have two kinds of securities in the market for possible investment choice:

(1) a risk-free security (e.g. a bond), where the price

 

0

S t at time t is given by

 

   

 

0 0 0

dS t  t S t t Sd , 0 0,

here 

 

t is a deterministic function;

(2) a risky security (e.g. a stock), where the price

 

1

S t at time t is given by

 

   

1 1 1 1

dS t  t S t td  t S t W t Sd , 0 0,

here W

 

. is a one-dimensional Brownian motion and

   

t , t 0

   are deterministic functions with

 

t

 

t

  .

Let π

 

t be a portfolio for the investors in the market, which is the proportion of the wealth invested in the risky security at time t.

Given the initial wealth _Yπ

 

₀ _{ }_y ₀_{, we assume} that π

 

. is self-financing, which means that the corre-sponding wealth process _Yπ

 

_. _{admits the following} dynamics

 

   



 



 

     

 

π π

π

d π d

π d ,

0 .

Y t Y t t t t t t

Y t t t W t

Y y

  



 

 _   _

      

(17)

A portfolio π is admissible if it is an _t-adapted process and satisfies

 



 



 



   



0

2 2

π

π d , - . .

T

t t t t

t t t P a s

  



 

  



(6)

 

   

dQ  Z T dP  , on T, (18)

where

 

   

 

d d ,

0 1.

Z t Z t t W t

Z

 



    

 (19)

We assume that

 

2

0 d , - . .

T

t s P a s

  



(20) If  





 

t



satisfies

 

1,

E Z T_ _ _ (21)

then Q_ is a probability measure. If in addition,

   

t t

 

t

 

t t,

 

0,T ,

     (22)

then

 

   

dQ  Z T dP 

is an equivalent local martingale measure. But here we do not assume that (22) holds.

All  satisfying (20) and (21) are called admissible controls of the market. The family of admissible controls

 is denoted by .

The problem is to find



_*_,_π*



_{  }_{such that}

 





 



*



*

π

inf sup

,

U Y T

E U Y T

E _



 _

 _ _

 

 

 _ _





 

 



Q

(23)

where U: 0,



   





,



is a given utility function, which is increasing, concave and twice continuously differentiable on



0,



.

We can consider this problem as a zero-sum stochastic differential game between the agent and the market. The agent wants to maximize his/her expected discounted utility over all portfolios π and the market wants to minimize the maximal expected utility of the agent over all “scenarios”, represented by all probability measures



Q_; 



.

To put the problem in a Markovian framework so that we can apply the dynamic programming, define















*



*

π

1 2

π

, , inf sup

,

V t x x U Y T t

E U

E

Y T t



 _  

 _  _

 

 

 

 

 

 

 

Q

(24)

where t

 

0,T denote the initial time of the investment, and x1X t x1

 

, 2 X t2

 

are the initial values of the

process

 

,π

 



 



2

1 2

: : ,

X s  X s  X s X s R given

by

 

   



 



 

   

     

 

 

1

π 2

π

d d

d

d d

0

d

π

d , , ,

π

Z s X s

X s

X s Y s

s

Y s s s s s

Z s s

W s s t T

Y s s s



  

 

 

   

   

   

 

  _ _

 

 _ _

 



 

  

 

 

(25)

which is a 2-dimensional process combined the Ra- don-Nikodym process Z

 

. with the wealth process

 

π _.

Y .

4.1. Maximum Principle Approach

To solve our problem by maximum principle approach, that is, applying Lemma 2.1, we write down the Hamilto- nian function (5) as





 



 



 

1 2 1 2 1 2

1 1 2 2

2 2

, , , ,π, ,, , ,

π π .

H s x x p p q q

x q x s s s p

x s q



   



 

   _   _



(26)

The adjoint Equations (6) are

 

   

 



 



1 1 1

1 2

1

d d d ,

,

p s s q s s q s W s

U

p T X T

x



 

 

 

   

(27)

and

 



 



 



 

     



 



 



2 2

2

d π

π d d

.

p s s s s s p s

s s q s s q s W s

U

p T X T

x

  



 

  _   _

 

 

 

     

(28)

Let

 

ˆ ˆ,π be a candidate optimal control (saddle point) and let Xˆ

 

. 



Xˆ1

 

. ,Xˆ2

 

.



be the corresponding state process, with corresponding solution

 



1

 

2

 



 



1

   

2



ˆ . ˆ . ,ˆ . , .ˆ ˆ . ,ˆ .

p  p p q  q q

to the adjoint Equations.

By (7) in Lemma 2.1, we first maximize the Hamilto- nian function H over all π . This gives the following condition for a maximum point ˆπ:

 



 



 

   

2 2 2

ˆ _ˆ _ˆ _0.

X s _ s  s p s  s q s _{ } (29)

Then, we minimize H over all  , and get the Following condition for a minimum point ˆ:

   

1 1

ˆ _ˆ _0.

X s q s

  (30)

(7)

 



   



1 ˆ2

ˆ ,

p s U f s X s (31)

with a deterministic differential function f . Differentiating (31) and using (17), we get

 

   



   



       



   



   



 



 



 



   



       



   



 

1 2 2

2 2 2 2

2 2

2

2 2

ˆ ˆ

d

1 _ˆ _π_ˆ _ˆ

2

ˆ _π_ˆ

ˆ _d

ˆ _π_ˆ ˆ _d _.

{

}

p s f s X s U f s X s

f s X s s s U f s X s

X s s s s s

f s U f s X s s

X s s s f s U f s X s W s



  



 



 

  



 

(32)

Comparing this with the adjoint Equation (27) by equat- ing the d ,ds W s

 

coefficients respectively, we get

 

       



   



1 ˆ2 ˆ2

ˆ πˆ ,

q s X s  s s f s U f s X s (33)

and

   



   



       



   



   



 



 



   





   

2 2

2 2 2 2

2 2

2

2 1

ˆ ˆ

1 _ˆ _π_ˆ _ˆ

2

ˆ _π _{( )}

ˆ

ˆ _ˆ _.

f s X s U f s X s

f s X s s s U f s X s

X s s s s s f s U

f s X s s q s



  



 

 



  

 

(34)

Substituting (33) into (30), we have

         



   



1 2 2

ˆ ˆ _π_ˆ ˆ _0,

X s X s  s s f s U f s X s

  (35)

or

 

 

ˆ

π s 0,s t T, . (36) Now, we try a process p sˆ2

 

of the form

 

   



   



2 ˆ1 ˆ2

ˆ .

p s  X s f s U f s X s (37)

Differentiating (37) and using (17), (36), we get

 

   





   



     



   



     



   





     



   



 

2 1 2

2 2

2

1 2

ˆ ˆ

ˆ d

ˆ ˆ

ˆ ˆ _d

ˆ ˆ ˆ _d _.

p s X s f s U f s X s

f s f s X s U f s X s

f s X s s U f s X s s

f s s X s U f s X s W s





 



 



 

 

(38)

Comparing this with the adjoint Equation (28), we have

 

     



   



2 ˆ1 ˆ ˆ2

ˆ ,

q s  f s X s  s U f s X s (39)

and

 



   



   



   



   



   





 

   



2 2

ˆ ˆ

ˆ ˆ _.

f s U f s X s s f s U f s X s

f s X s U f s X s f s s f s



   

 

  (40)

Substituting (39) into (29), we have

 





   



   



       



   



1 2

ˆ ˆ

ˆ

ˆ ˆ _0,

s s X s f s U f s X s

s f s X s s U f s X s

 

 

 



 

or

   

_s ˆ _s

 

_s

 

_{s s}_,

 

_{t T}_, _.

     (41) From (40), we get

   





   



   







 

   





2 2 2

ˆ ˆ ˆ

0,

U f s X s X s f s U f s X s

f s  s f s

  



  

or

 

   

0,

f s  s f s  (42) i.e.,

 

exp



T

 

d ,



 

, .

s

f s 

_

 r r s t T (43)

Let t0, we have proved the following theorem.

Theorem 4.1 The optimal portfolio strategy ˆπ 

for the agent is

 

 

ˆ

π t 0,t 0,T . (44)

The optimal “scenario”, that is, the optimal probability measure for the market is to choose ˆ   such that

 

1



 



 

ˆ _t _t _t _t _,_t _0,_T _.

    

   (45)

That is, the market minimize the maximal expected utility of the agent by choosing a scenario (represented by a probability law dQ_ Z T_

 

dP), which is an equivalent martingale measure for the market (see (22)).

In this case, the optimal portfolio strategy for the agent is to place all the money in the risk-free security, i.e., to choose π

 

t 0 for all t.

This result is the counterpart of Theorem 4.1 in An and Oksendal [6] without jumps with complete information.

4.2. Dynamic Programming Approach

To solve our problem by dynamic programming approach, that is, applying Lemma 2.2, we write down the generator

,π

A of the diffusion system (25) as









 



 



 





 





   





     





,π

1 2 1 2

2 1 2

2 2

1 2 1 2

1 2

2 2 2

2 2 1 2

2 2

1 2 1 2

1 2

, , , ,

π , ,

1 _{, ,}

2 1

π , ,

2

π , , ,

A t x x t x x

t

x t t t t t x x

x

x t t x x

x

x t t t x x

x

t t t x x t x x

x x

 _ 



  

 

 



 

 



 

 _   _{ }

 

  

  



(8)

for _

 

_.,.,. __C1,2,2



 

_0,_T __{R R}2_;



_.

Applying to our setting, the HJBI Equation (10) gets the following form

 









 









 









ˆ

,π 2

1 2 1 2

ˆ,π 2

1 2 1 2

ˆ ˆ,π 2

1 2 1 2

, , 0, , , ,

, , 0, π , , ,

, , 0, , .

i A V t x x x x R

ii A V t x x x x R

iii A V t x x x x R



    

    

  

(47)

We try a V of the form



, ,1 2



1



 

2



,

V t x x x U f t x (48)

for some deterministic function f with f T

 

1. Note that conditions (i), (ii), (iii) in (47) can be rewritten as





ˆ





ˆ ˆ

,π ,π

1 2 1 2

infA V t x x , , A V t x x , , 0,

   (49)









ˆ,π ˆ ˆ,π

1 2 1 2

π

supA V t x x , , A V t x x , , 0.

   (50)

Maximizing A V t x xˆ,π



, ,1 2



over all π gives the following first-order condition for a maximum point ˆπ:

 

     



 



     



 



2 2

ˆ 0

ˆ

π .

t t t t U f t x

x t t f t U f t x

   



  

_   _



 (51)

We then minimize ,πˆ





1 2 , ,

A V t x x over all  and get the following first-order condition for a minimum point ˆ:

 

1 2

   



 

2



ˆ

π t x x  t f t U f t x 0.

  (52)

From (52) we conclude that

 

 

ˆ

π t 0,t 0,T , (53)

which substituted into (51) gives

 

_t

 

_t ˆ

   

_t _{t t}_,

 

_0,_T _.

     (54)

And the HJBI Equation (iii) ˆ ˆ,π





1 2

, , 0

A V t x x  states that with these values of ˆπ and ˆ, we should have

 





 



 



 

1 2 2 2 1 2 0,

x U f t x x f t  x  t x U f t x f t 

or

 

   

0,

f t  t f t  (55) i.e.,

 

exp



_tT

 

d ,



 

0, .

f t 

_

 r r t T (56)

We have proved the following result.

Theorem 4.2 The optimal portfolio strategy πˆ 

for the agent is

 

 

ˆ

π t 0,t 0,T (57) (i.e., to put all the wealth in the risk-free security) and the

optimal “scenario” for the market is to choose ˆ 

such that

 

1



 



 

ˆ _t _t _t _t _,_t _0,_T

 _   _ _

(58)

(i.e., the market chooses an equivalent martingale measure or risk-free measure Q_ˆ for the market).

This result is the counterpart of Theorem 2.2 in Ok- sendal and Sulem [15] without jumps.

4.3. Relationship between Maximum Principle and Dynamic Programming

We now verify the relationships in Theorem 3.1. In fact, relationship (13), (14), (15) is obvious from (47). We only need to verify the following relations

 





1

1 2 2

2

ˆ

, , ,

ˆ

V

p t _x

t x x V

p t

x 

 

_ 

 

   _ 

 













and

 





   

 

2 2

2

1 1 1 2 1

1 2

2 2

2 2 1 2

ˆ ˆ

, , .

ˆ πˆ

V V

q t x x x x t

t x x

q t V V x t t

x x x

 

   

  _ _

  _   _ 

 



 _{ } _

  _ _  

  _ _  

_ _ 

 

Note that πˆ

 

t 0, the above relations are easily obtained from (48), (31), (37), (33) and (39).

5. Conclusions and Future Works

In this paper, we have discussed the relationship between maximum principle and dynamic programming in zero- sum stochastic differential games. Under the assumption that the value function is smooth, relations among the adjoint processes, the generalized Hamiltonian function and the value function are given. A portfolio optimization problem under model uncertainty in the financial market is discussed to show the applications of our result.

(9)

6. Acknowledgements

This work is supported by National Natural Science Foun- dations of China (Grant No. 11301011 and 11201264) and Shandong Province (ZR2011AQ012).

REFERENCES

[1] R. J. Elliott, “The Existence of Value in Stochastic Dif- ferential Games,” SIAM Journal on Control and Optimi- zation, Vol. 14, No. 1, 1976, pp. 85-94.

[2] W. H. Fleming and P. E. Souganidis, “On the Existence of Value Function of Two-Player, Zero-Sum Stochastic Differential Games,” Indiana University Mathematics Journal, Vol. 38, No. 2, 1989, pp. 293-314.

[3] R. Buckdahn, P. Cardaliaguet and C. Rainer, “Nash Equi-librium Payoffs for Nonzero-sum Stochastic Differential Games,” SIAM Journal on Control and Optimization, Vol. 43, No. 2, 2004, pp. 624-642.

[4] S. Matatamvura and B. Oksendal, “Risk Minimizing Portfolios and HJBI Equations for Stochastic Differential Games,” Stochastics: An International Journal of Prob- ability and Stochastics Processes, Vol. 80, No. 4, 2008, pp. 317-337.

[5] S. J. Tang and X. J. Li, “Differential Games of N Play- ers in Stochastic Systems with Controlled Diffusion Terms,” Automatic and Remote Control, Vol. 69, No. 5, 2008, pp. 874-890.

[6] T. T. K. An and B. Oksendal, “Maximum Principle for Stochastic Differential Games with Partial Information,”

Journal of Optimization Theory and Applications, Vol. 139, No. 3, 2008, pp. 463-483.

[7] A. Bensoussan, “Maximum Principle and Dynamic Pro- gramming of the Optimal Control of Partially Observed Diffusions,” Stochastics, Vol. 9, No. 3, 1989, pp. 169-222.

[8] X. Y. Zhou, “The Connection between the Maximum Principle and Dynamic Programming in Stochastic Con- troll,” Stochastics & Stochastics Reports, Vol. 31, No. 1-4, 1990, pp. 1-13.

[9] J. M. Yong and X. Y. Zhou, “Stochastic Controls: Ham- iltonian Systems and HJB Equations,” Springer-Verlag, New York, 1999.

[10] N. C. Framstad, B. Oksendal and A. Sulem, “A Sufficient Stochastic Maximum Principle for Optimal Control of Jump Diffusions and Applications to Finance,” Journal of Optimization Theory and Applications, Vol. 121, No. 1, 2004, pp. 77-98.

[11] J. T. Shi and Z. Wu, “Relationship between MP and DPP for the Optimal Control Problem of Jump Diffusions,”

Applied Mathematics and Optimization, Vol. 63, No. 2, 2011, pp. 151-189.

[12] C. Donnelly, “Sufficient Stochastic Maximum Principle in a Regime-Switching Diffusion Model,” Applied Ma- thematics and Optimization, Vol. 64, No. 2, 2011, pp.

155-169.

[13] G. Peskir and J. Shorish, “Market Forces and Dynamic Asset Pricing,” Stochastic Analysis and Applications, Vol. 20, No. 5, 2002, pp. 1027-1082.

[14] R. Korn and O. Menkens, “Worst-Case Scenario Portfolio Optimization: A New Stochastic Control Approach,”

Mathematical Methods of Operations Research, Vol. 62, No. 1, 2005, pp. 123-140.

[15] B. Oksendal and A. Sulem, “A Game Theoretical Ap-proach to Martingale Measures in Incomplete Markets,”

Pure Mathematics, E-Print of University of Oslo, No. 24, 2006.

[16] B. Oksendal and A. Sulem, “Forward-Backward SDE Games and Stochastic Control under Model Uncertainty,”