Viscosity solutions associated with impulse control problems for piecewise deterministic processes

(1)

Internat. J. Math. & Math. Sci.

VOL. 12 NO. 1 (1989) 145-158 145

VISCOSITY

SOLUTIONS

ASSOCIATED

WITH

IMPULSE CONTROL PROBLEMS

FOR

PIECEWISE-DETERMINISTIC

PROCESSES

SUZANNEM.LENHART

UNIVERSITY

OF

TENNESSEE

MATHEMATICS DEPARTMENT

KNOXVILLE,

TENNESSEE

37996-1300

(Received

July 1, 1987 and inrevisedform

Janurary

5,

1988)

ABSTRACT.

Thispaper considersexistenceand uniqueness results for viscositysolutions ofintegro-differentialequationsassociatedwiththe impulse controlproblemfor piecewise-deterministic processesonbounded domains andon

R".

KEY WORDS

AND

PHRASES.

Impulse control,viscosity solution, piecewise-deterministic processes.

1980AMS

SUBJECT CLASSIFICATION CODE.

35F20

1. INTRODUCTION.

This paper considers viscosity solutions ofintegro-differentialequationsassociated with the imptflsc controlproblemfor pieccwise-deterministic

(PD)

processes

max(Lu

f,

u

M

u)

0in

E

(1.1)

where

Lu(z)

E

g,(z)uz,(x)

+

a(z)u(x)

$(x)

IS

(u(y)

u(x))

Q(dy,

x)

Mu(x)

inf_(u(x

+

)

+

c()).

(2)

146 S.M. LENHART

We consider the cases when

E

ft, a bounded domain in

R",

and

E

R". In

the bounded domain case,wehavethe following

boundary

condition:

u(x)

jf

u(y)Q(dy,

x)

for allxOfL

(1.2)

Let

usbriefly givethe

background

for thisproblem.

A

PD

process,

x(t),

with jump rate,

(x),

and jump distribution

Q(dy,

x),

follows deterministic dynamics between random jumps:

dx(t)

(9,

(x(t))

(x(t))).

dt

Davis

[1]

developed theprobabilisticsideof these

PD

Markov processes. Ifthe ith jump ofthe processoccursat

T,

thenthedistributionof

x(Ti)

is

Q (dy,

z(T-))

and

(/o’

)

P(+

T,

>

)

((,

+

))d

Davis

[1]

showed that a

PD

processisastrong Markovprocess withgenerator

i----I

with

E,

the state spe. The undy conditioncs

cause

the

PD

press jumps baintothenteiorof

,

un

hittin$theundyof

.

The jumps

Td

epartof the prs,

x().

Csi&r whencertnjumps, "imputes’, econtrolled fromoutside the

PD

process.

Supse

the stateiscgedfromx tox

+

withimpul

O,

d cost

t()

isincurredwhentheimpulse

s

appli.

An

impul control

strate

visasequence ofstoppingtimd impuls,

(a,

,

a,

,...

},

(a

).

The

contro PD

press x"satisfi

z’(0,

+

0)

x(O,

0)

+

.

(1.3)

Theiat ctnctionis

TheminimM cost functionis

V()

if

J,(,).

(.4)

HeuristicMly,the dynamicproingequation satisfiedbythe minimcost function is

i

y

(1.1).

(3)

VISCOSITY SOLUTIONS OF INTEGRO-DIFFERENTIAL EQUATIONS 147

In

thispaper,wedefinethe notionof viscosity solution of

(1.1)

and proveexistence and uniquenessresults in the viscosity sense. The control representation is discussed in the last section.

2.

UNIQUENESS.

Theoriginal

formulations

of viscosity solution

definitions,

by

Crandall

and Lions

[9,

10],

did not includeintegralterms inthe operators,so westate thedefinitionextensionto this case.

DEFINITION:

tteBUC(")

(bounded,

uniformly

continuous)

(E

willbe for

R")

(i)

uisaviscosity subsolutionof

(1.1)

if

m

(-,(o),,,

(o)

+

’(:o),(o)

/E

(u(V)-

U(Xo))Q(dy, xo)

f(xo),

A(Xo)

(o)-

M(o))

o

(2.)

whenever

eC

(E)

du haglobMmimumat

xo.

(ii)

uisaco,ityauperaolutionof

(1.1)

if

=

(-a(o),, (o)

+

(o )(o

f

((v)- (o))(dv,

o)-/(o),

A(xo)

(o)-

M(o))

0

(2.2)

whenever

eC

(E)

du haglobMminimumat

xo.

Note

that implicit sumtion on

reated

_subscripts is usedontheg, terms above.

We

me

the

followg

smptions:

A,

f,

a_{bounded uniformly}_continuous_on

E,

gi Lipschitzcontinuouson

E,

1,...,

a(z) >_

co >

0on

E,

A(x)

_

O,

f(x) >

0on

E,

for each fixed

xE,

Q(dv,

x)

isaprobabilitymeasure whichisLipschitz continuous as afunctionofx, i.e.,

/E

()(d’x)

-/E

()0(dv, z) <

CIIIILooI

zl

for all

OCL(E),

(2.7)

(2.3)

(2.4)

(4)

I8 _S.H. _LT

c({)

--

ooas

{

-4o,

(0)

,

c({)>_ k>Oforall_>O.

We

assumethatfisasmoothboundeddomainin

R".

We

needanassumption thatguarantees

c(n)

Mc(a),

fromLionsand Perthame

[11],

wehavetheneededassumption:

for

allzd,

{{>0l{#0,

z+’t0Q,

9s>O, s.t.Vy>0,

(2.0)

z+!/Cif[y-[<}

isempty. If

D

isconvex, then

(2.10)

holds.

We

nowprovecomparison resultsinflandthenin

R"

that yield uniqueness rcsults for equation

(1.1).

TXEOrtEM 2.1. Under assumptions

(2.3)-(2.10)

on

l,

ifu is a viscosity subsolution of

(1.1)

on andv isaviscositysupersolutionof

(1.1)

on with

u(z)

<

n

u(y)Q(dy,

z)

and

v(.)

>_

n

v(y)Q(dy,

z)

/orall xE

OD,

(2.11)

thenu

<

v on

D.

PROOF:

Let

0

<

#

<

1 and setw #u. Thereexistszin

D

such that

(w

v)(z)

m_ax(w

v)(x).

Firstweshowthatwe canchoosez sothat z Off. IfzE

OQ,

then

m(- )

(,)

<

(()-

())

Q(a,

)

by

(.**)

<

sup(w

v).

If the maximum ofw- vdoesnotoccurataninteriorpoint, then thereexistsaset

A

C

D

such that

Q(A,

z) >

0 and

w(y)- v(y)

<

max(w

v)

for all

yeA.

Then

Q(dy,

z)

<

sup(w-

v),

v(y))

which isacontradiction. Sothereexists wherethemaximum occurs. Thuswecan assume z(

t.

(2.s)

(5)

VISCOSITY SOLUTIONS OF INTEGRO-DIFFRRBNTIAL EQUATIONS 149

Then

st

--

I111-,

I1

II-).

,(= ,)

,,,(=)

()

=

-

C,l

where

C,

,

(’)

d

,

is a mulus of continty for v.

(=’, ’)

fl

su

that

There exists

implies

="

-

"

=

+

c,I/-

zl

=

_<

w(=’)

,,(=’)

(,(z)

,,(=))

+

,,(=’)

<

2M.

Refiningthisestimategives

andthen

+

o,1,"

_<

,,.,,,

(1="

u’l)

Thisimplies

OascO.

Also

C,l,"

=1

=

<

c,

=,

and then

lY"

zl

--"0ase: O.

Noticealsowehave for esufficiently small,x*, y* E since zE ft. Sincewisaviscosity subsolution of

max

(-giw.,

+

aw )

jf

(w(y)

w(x))

Q(dy,

x)

#f

w

.-u,)

=0,

Mw(x)

inf_(w(x

+

)

+

#C())

e>0

Ce

ly

zl

2 hasamaximumat

z’,

(2.12)

(6)

150 S.M. LENHART

_g.(=,)

l:

(="

v"

,(,)_ w()

_<

o.

(2.14)

Sincev is a

supersolution

of

(1.1)

and

---.

,,(v)-

((o)

hasaminimum aty,

I’

)

xo

-_____v

C,

ItJo

zl

=

max

(-e.(!/’)

(2

(

a:"

e-

!/).

+

2C,

(/*

+

a(y’)v(y’)-

A(y’)

fn

(v(!/)

v(!/’))Q(dy,

y’)

v(y’)- My(y’)) >

O.

CAS

A.

,,(v’) >_

There exists

*

>

0 such that

Mv(z)

v(x"

+

’)

+

c(’).

From property

(2.10),

My6_.

C(fl),

and

Mv(x)

Mv(!f)

canbemade smallfor small.

c,

Iv"

1

-C,

lv’-

+(=’)-(’).

u

(-

z)(’)

_< (- z),

(I,,(=’v’) _<

=(="

+

’)

,(="

+

’)

C,l="

+

zl

+

c.

(I"

+

’-

,l’

-I"

t

)

+

(.

-

)

+

Mv(:r,’)

Mv(y’).

Hence

for smallenough,using(p ]k

<

O,

whichcontradictsourchoiceof

(z

,

V’)-Notice this partiswhere

C,

isused.

CASE B:

v(!/) <

My(v*).

(7)

VISCOSITY SOLUTIONS OF INTEGRO-DIFFERENTIAL EQUATIONS 151

-<

2

(9’(Ye)

9’(=e))

(2.16)

2g,(y’)C,(y"

z)i

(y’)/o

(v(y)

v(y’))

(du,

y)

+

x(=’)

((u)-g

c(1-

#)

+

o,(1).

(Note

that

o,(1)

0

0.)

S

nhEt

[12]

for estimateonsuchintegral terms; thekey ideausedisthat

[(w(y)

v(y))

(w(z)

v(z))]

Q(dy,

x

e) <_

0

(2.17)

and theintegralterms in

(2.16)

areclose tothe integralterms in

(2.17).

Thusweconclude max

(w(z)

v(x)) < C(1

#)

+

oe(1).

Let/z-,l,

max

(u(x)

v(z))

<

oe(1).

Then lete O,

max(u(x)

v(z)) <

O. |

Nowweobtainasimilarresult in

R".

THEOREM 2.2. Under assumptions

(2.3)-(2.9)

on

R",

ifu is a viscositysubsolution of

(1.1)

on

R"

andv isaviscositysupersolutionof

(1.1)

on

R",

then

,,() _<

(=)

o.

R".

PROOF"

Let

o

<

$

<

1. Usingnotationfromproofof Theorem2.1, choosezsuch that

w(z)

v(z)

>_

sup(w

v)(z)

.

If

Cely-

zl

2

+

Il

> 5mx(ll=ll, Ii,11),

with

Ce

V/W

(v/2M + 1),

,(=,

v) _< ,(,,)

(Xe,ye*).

Thus

qe

doesachieve its maximum atafinitepoint,say at (xe,y

*)

(x**,

ye*).

Weobtain

(2.12)-(2.15)

asbefore,with xe

yes

.._,

0as --,0and 6 O.

(8)

152 _S.M. _LENHART

and

I]1 < C1

where

C1

dependson

IIt,

lloo

but noton

e,6. Now

_< ("

+

’)

+

m(

)

(.,

+

’)

(’)

+

k(1

,)

-<

,("

+

’)

,,(v"

+ T’)

,

c,

Iv"

+

]

+ c, (I/+

’

1

-I/-

1) +

-(,

)

_<

q,(x"

+

],y"

+

(])

+

o,,(1)

+

2k-(#

1)

<

,(x"

+

],

y"

+

])

foresufficiently small. Thiscontradicts thechoiceof

(x’,y’).

CASE B:

v(lf) <

Mv(y

e)

follows asinTheorem 2.1. |

3.

EXISTENCE

RESULTS.

Due

to the possible incompatibility of the impulse obstacle and boundary condition

(1.2),

weshallproveexistenceof viscosity solutions to

max(Lu

f,

u

M

u)

0 infl satisfyingtheboundarycondition:

u(z)

Mu(z)

A

L

u(y)Q(dy,

z)

forzeOft

(A

minimum

symbol).

(3.2)

Condition

(3.2)

formallymeans thestate process couldjump back intotle interior of ft uponhitting

o

oranimpulse couldbeusedtochange the state.

We

havethe following

existenceresult.

THEOREM 3.1. Under assumptions

(2.3)-(2.10)

on

f,

thereisaunique viscositysolution of

(3.1)

onft satisfying

(3.2).

(9)

VISCOSITY SOLUTIONS OF INTEGRO-DIFFERENTIAL EQUATIONS 153

max(Lu

f,

u

)

0on12

(3.3)

satisfying

A

L

u(y)Q(dy,

x)for ze0f/.

(3.4)

We

alsohave thecontinuous

dependence

ofu on

,

i.e.,iful,u2 areviscosity solutionsof

(3.3)

satisfying

(3.4)

withobstacles

1,

2, respectively,

II,=

,=Iloo _<

II,

=Iloo.

Using this continuousdependence result, we obtain theexistence and uniqueness of vis-cosity solutions of

(3.3)

satisfying

(3.4),

forobstacles in

C(12).

Now

we willapply this

resultwith

Mu,

with

uC(12)

giving

MuC(12)

by

(2.10).

We

nowconstructasequence which will convergetothesolution of

(3.1)

satisfying

(3.2).

Defineu0 to bethe unique viscositysolutionof

Luo

f

in

R

u0(z)

/

u0Cy)QCdv,

z)

onOf/.

By

(2.10),

u0eC(12)

implies

Mu0eC(12)..

Thus thereexistsasequence of viscositysolutions, satisfying

max(Lu,,

f,

u,,

Mu,-l)

0in 12

,.()

M,,._()

^

[

.(v)Q(dv,)

o

On.

do

By

amaximumprincipleargumentonthese viscositysolutions, weobtain

0

_

u,,

_

u._l, n 1,2,

To get

uniform convergenceofoursequence,first defineamap

cr"

C(12)

--,

C(12)

by

()

uwhere

uisthe viscosity solution of

(3.3)

satisfying

(3.4).

(10)

154 S.H. LENHART

If thereexists

e[0,1]

such that

v w

_<

w,

then

Tv

Tw

<_ (1

A)/Tu.

(3.5)

See Hanouzet

and Joly

[13],

Perthame

[14,

15]

and Barles

[8,

16]

for examplesofthis

technique. Siuce u0 ul

_

-0, veiterate

3.5)

toobtain

In

this way, weobtain uniform convergence, u,, --, _u. _The uniform convergence ofu, insures the convergence of the integral termsand Mu,_l, and wehave that u satisfies

boundary

condition

(3.2).

To

showthelimitfunctionuisaviscositysolution,wefirstlookatpointx0 where

u-hasa

global

minimum. Then thereisasequence

{x,

}

convergingtox0 such that u,

hasitsminimumat x,. Sinceu,

<

Mu,,_l on

fl,

weknow

u

<

Mu

onf.

Ifu

Mu

atx0,then theviscositysupersolutioncondition

(2.2)

issatisfied. Ifu

<

Mu

atx0,

then

u.(x.) < Mu._(x.)

forn large

enough,

which implies

A

lft

(u.(y)- u.(x,,))Q(dy,

x.) >_

f

at

x..

Letting n

,

we haveu is a viscosity

surlution.

The sublution c follows similly.

Theuniquene resultfollows inThrem2.1except forthecewhen themaximum

ofw vcursat

zeO.

v(z)

v(y)Q(dy,x)

then thegumentg fore.

v(z)

My(z),

thenthereexistsnonzero 0

su

that

My(z)

v(z

+

)

+ c().

Then

,(z)

() <

,(

+

)

+

,()

(

+

p)

()

< ,(

+

)

(

+

)

+

(

)

< (

+

)

(z

+

(11)

VISCOSITY SOLUTIONS OF

INTEGRO-DIFFERENTIAL

EQUATIONS ₁₅₅

The existence

proof

for the

R

caseissimpler.

THEOREM

3.2. Underassumptions

(2.3-2.9

on

R",

there

ests

aunique viscosity so-lution to

(1.1)

in

R

n.

PROOF" We

usetheiterative approximationschemeon

R

n,

max(Lun

f,

u,

Mun_)

0 in

R

n,

n 1, 2,... and

Luo

f

in

R

n.

(3.6)

To

obtainexistencefor

(3.6),

we useextensionsof results from

[5, 12]

with

b

in

BUC(Rn).

Note

that

un_1eBUC(R

n)

implies

Mun_IeBUC(R).

By

usinganoperator

T

asin

The-orem

3.1,

weobtain uniform convergence of

{un}

toaviscositysolutionu. The uniqueness resultisTheorem2.2. |

4.

CONTROL

REPRESENTATION.

To

put the results obtainedin section 3in thecontextof theclassicalresultsonimpulse

control

(Bensoussan

and Lions

[17]

and Menaldi

[18])

weshow that thesolution obtained inTheorem3.2isequaltothevaluefunction associatedwiththe impulsecontrolproblem

(1.4).

THEOREM

4.1. Theunique viscosity solution

u(z)

from Theorem3.2isequa/tothe value function

V(x)

from

(1.4)

PROOF" First,weshow that the approximations

un

have thefollowingcontrol interpreta-tion:

un(x)

inf{J.(v,)’vn

impulsecontrol strategywith 8,

,

for all

_>

n

+

1}, (4.1)

i.e.,

u

isthe minimum cost function associated withthe impulsecontrolproblemwithat most nimpulses allowed.

We

show

(4.1)

byinduction. Call the right hand side of

(4.1),

Vn(x).

The representation foru0

(no

impulses)isvalid.

Assume

(4.1)

forUn-:.

By

[5,12],

wehave

Un(X)

if

E

J’(z())e-’d

+

MUn_:(.(O))e

-’

(4.2)

where0is astoppingtime.

Let

Vn be animpulse control

strategy

with n impulses,

v.

(01,6,...,0,,,).

(12)

156 S.M. LENHART

Therefore

..() <_ v.().

To

show

u.

_> Vn,

for

>

0,choose8, such that

(4.3)

Using

(2.8),

choose

1

such that

M.._,(.(O,

0))

,,._,(.CO,

o)

+

,)

+

(,).

(4.4)

By

the inductive hypothesison

u.-,(z($1-0)+),

thereexistsanimpulse control

strategy

v._]

(S,,,,...

O

>

01,

uchthat

(4.5)

where

v.

(0,,

,,

v._]

).

Thiscompletestheproof of

(4.1).

By

construction,

u.

u. Thus, by

(4.1),

u(z)

<

inf

Jz(v.)

for alln.

(13)

VISCOSITY SOLUTIONS OF INTEGRO-DIFFERENTIAL EQUATIONS 157

,,(,) _< J,(.)

<

v(.)

+

e.

Thusu

_<

V. To

showu

_> V,

thereexistsindexj

large

enough

and

strategy

v,

such that

,()

<

.()

+

12

<

.()

+

andhence

V(z)

_<

u(z).

|

Similar control_{representations can} _{be obtained}_in_the

bounded

domaincase.

ACKNOWLEDGEMENT.

The researchwas

supported

inpart

by

NSF

grant

#DMS-8508651,

Institutefor Mathematics and

Its

Applications, Minneapolis,and University of

Tennessee

Science Alliance

Research

Incentive

AwaruL

REFERENCES

Z.’:,qs,,i.H. A.,P/eceudae-determ/n/iicMarkov process:A

gene

cls_oI_non-sionmeb, J.

y

Ststt. B.46(19),

2.DAVIS,M. H.A., ontl_ofee-deingic

a

e-gime_amic_{Pmming, in}

"Pn

ofSBHnef SymiumonShti DiffetiSystem," I85.

.

VERM, D., timntl_ofpie.deiinticMaovp,e, Sthti14 (1985),

1.

4.NER, M., fimdConfl Stafe-SpaCotinf II,SIAMJ.dntl dtim.24

5.LENHA,S.M.dLIAO,Y. C.,Inte-dinfi afiommsodafed thopfimalafoppin9

me of

apie-detemintieps, ShtiIS(1985),183-207.

e.

LENHA,S. M.dLIAO, Y. C.,5tchingUontl_ofe.DeleminticPse,Jour

d Optim._Thawd Appikations(toap).

7.GUGERLI,U.S., timal$ZoppingofaPiee-DeleminticMaroPess, Sthti19 8.BARLES, G.,Deiemintic Impbive ContmlPble, SIAM J.of ntroldOptimization(). 9.CRANDALL, M. G., EVANS, L.

.

andLIONS, P. L., Somepiesofvcosit solut,o of

Hamilton-Jacobiealions,

sns.

A.M.S.282(1984),487-502.

10.CRANDALL, M.G. dLIONS, P.L., VcositSolutions_ofHamilton-acobi Equation,

s.

A. M.S.93(1983),14.

11.LIONS, P. L. and PERTHAME, B., asi-aalionalInequahttes and Ergodic Impue Contl,

SlAMJ.ofntrol and Optimization24(1986),604-615.

12.LENHART,S._{M., Vcosit}5olutioAssociated th Sching Contl Problefor

Piecewise-De-re--in,ticPcesses,Houston J.ofMath.(tospear).

13.HANOUZET, B. dJOLY, J. L., Conveenceunifoe des itrs dfinnisant lasolutwnd’une

inquationiaationelleabiraite, Compte Rend(Paris)286A(1978),735-745.

14. PERTHAME,B.,lnequalioQui-VaatwnnellesetEquations de Hamilion-3acobi-Bellmanda Rn,Ann. de Toulou5(1983),237-257.

15. PERTHAME,B.,Quasivaalional lnequahtzes and Hamilton-Jacobi-BellmanEquationsin aBounded Rion,

mm.

P. D.E.9(1984),561-595.

16.BARLES, G.,Qui-vadational inequalihesand_firstorder Hamilton-acobiequations,Ths, Uni-recitedeParIX-Dauphine(1984).

17.BENSOUSSAN,A.andLIONS, J.L.,,Impulse Control andQuasi-vaatzona lnequaht:es, Dun,