Stochastic singular optimal control problem of switching systems with constraints

(1)

R E S E A R C H

Open Access

Stochastic singular optimal control

problem of switching systems with

constraints

Charkaz Aghayeva

*

*_{Correspondence:} [email protected] Department of Industrial Engineering, Anadolu University, Eskisehir, Turkey

Institute of Control Systems, ANAS, Baku, Azerbaijan

Abstract

This paper is devoted to the optimal control problem of switching system in which constraints on the state variable are given by inclusions. Using Ekeland’s variational principle, second-order necessary condition of optimality for stochastic switching systems with constraints is obtained, and transversality conditions for switching law are established.

Keywords: stochastic optimal control; necessary condition of optimality; singular switching systems; transversality conditions

1 Introduction

Stochastic differential equations have provided a lot of interest for problems of nuclear fission, communication systems, self-oscillating systems,etc., where the influences of ran-dom disturbances cannot be ignored [, ].

Hybrid systems, including special class-switching systems, are widely use for representa-tion of noninvariant phenomena. Switching systems have the beneﬁt of modeling dynamic processes with continuous law of movement. For general theory of stochastic switching systems, we refer to [–]. Recently, optimization problems of hybrid systems and, in par-ticular, optimization problems of switching systems have attracted a lot of theoretical and practical interest [, –].

Stochastic control problems have a variety of practical applications in fields such as physics, biology, economics, management sciences,etc.[, ]. A classical approach for optimization and particularly for control problems is to derive necessary conditions satis-fied by an optimal solution. The modern stochastic optimal control theory has been devel-oped along the lines of Pontryagin’s maximum principle and Bellman’s dynamic program-ming [, ]. The stochastic maximum principle has been first considered by Kushner []. The earliest results on the extension of Pontryagin’s maximum principle to stochas-tic control problems are obtained in [–]. A general theory of the stochasstochas-tic maximum principle based on random convex analysis was given by Bismut []. Modern presen-tations of stochastic maximum principle with backward stochastic differential equations are considered in [–]. First-order necessary conditions of optimality for stochastic switching systems have been studied by the author in [–].

(2)

It is well known that the first-order necessary conditions provide a basic tool to study the properties of optimal controls. However, in some cases, the Hamilton function van-ishes almost everywhere, and therefore, first-order necessary conditions cannot provide enough information to find the desired optimal controls. The above-mentioned cases are called singular, and to investigate these situations, an additional test in terms of high-order necessary conditions is required. Deterministic optimization problems for singular con-trols were intensively used by [, ]. Such kind problems for stochastic systems have been investigated in [, ].

In this paper, the singular optimal control problem of stochastic switching systems with uncontrolled diffusion coefficients is considered. Second-order necessary conditions of optimality and transversality conditions are obtained in [] for uncontrolled switching systems. Ekeland’s variational principle [] has been widely used in various areas of anal-ysis such as fixed point analanal-ysis, optimization, and optimal control theory. In this paper, backward stochastic differential equations and Ekeland’s variational principle is used to establish a singular maximum principle for stochastic optimal control problems of con-strained switching systems.

2 Preliminaries and formulation of problem

Throughout this paper, we use the following notation. ByNwe denote some positive con-stant,Rn_{denotes the}_{n-dimensional real vector space,}_{| · |}_{denotes the Euclidean norm}

inRn_{, and}_E_{represents the mathematical expectation. Assume that}_w_(t),_w_{(t), . . . ,}_wr_(t)

are independent Wiener processes that generate the ﬁltrationFl

t=σ(w¯ l(t),tl–,tl),l= ,r,

 =t<t<· · ·<tr=T. Let (,F,P) be a probability space with ﬁltration{Ft,t∈[,T]},

whereFt=

r

l=Ftl;LF(a,b;Rn) denotes the space of all predictable processesx(t,ω)≡x(t)

such thatE_ab|xt(ω)|dt< +∞;Rm×nis the space of linear transformations fromRmtoRn.

LetOl⊂Rnl,Ql⊂Rml,l= ,r, be open sets. Unless speciﬁed otherwise, we use the

fol-lowing notation: t = (t,t, . . . ,tr), u = (u,u, . . . ,ur), x = (x,x, . . . ,xr).

Consider the following stochastic control system:

dxl(t) =glxl(t),ul(t),tdt+flxl(t),tdwl(t), t∈(tl–,tl],l= ,r, ()

xl(tl–) =l–

xl–(tl–),tl–

, l= ,r;x_t_=x, ()

ul(t)∈U∂l ≡

ul(·,·)∈L_Fl|ul(t,·)∈Ul⊂Rml, a.c.

. ()

Elements ofUl

∂are called admissible controls. The problem is to ﬁnd an optimal solution (x,u) = (x,x, . . . ,xr,u,u, . . . ,ur) and a switching sequence t = (t,t, . . . ,tr) such that the

cost functional

J(u) =

r

l=

E

ϕlxl(tl)

+

tl

tl–

plxl(t),ul(t),tdt ()

is minimized on the decisions of system ()-() generated by all admissible controlsU= U_×_U_{× · · · ×}_Ur_{under the conditions}

Eql_xl_(t l)

∈Gl_, _l_{= ,}_r, _()

(3)

Assume that the following requirements are satisﬁed:

(AI) The functionsgl_,_fl_,_pl_,_l_{= ,}_r_{, and their derivatives are continuous in}_(x,_u,_t)_.

(AII) The derivatives ofgl_,_fl_,_pl_,_l_{= ,}_r_{, are bounded by}_{N( +}_|x|)_.

(AIII) The functionsϕl(x),l= ,r, and the functionsl(x,t),l= ,r– , are continuously diﬀerentiable, and their derivatives are bounded byN( +|x|). (AIV) The functionsql_(x)_,_l_{= ,}_r_{, are continuously diﬀerentiable, and their derivatives}

are bounded byN( +|x|).

Furthermore, the following requirements are assumed.

(BI) The unctionsgl_,_fl_,_pl_,_l_{= ,}_r_{, are twice continuously diﬀerentiable with respect}

tox.

(BII) The functionsgl_,_fl_,_pl_,_l_{= ,}_r_{, and all their derivatives are continuous in}_(x,_u)_;_gl x,

gl

xx,fxl,fxxl,plxxare bounded.

(BIII) The functionsϕl(x) :Rnl→_R_,_l_{= ,}_r_{, and the functions}l_(x,_{t) :}_Rnl×_T→_R_,

l= ,r– , are twice continuously bounded diﬀerentiable.

(BIV) The functionsql(x) :Rnl→_R_,_l_{= ,}_r_{, are twice continuously bounded}

diﬀerentiable. Consider the sets:

Ai= Ti+× i

j=

Oj× i

j=

Qj, i= ,r,

with the elements

πi=t,t,ti,x(t),x(t), . . . ,xi(t),u,u, . . . ,ui

.

Deﬁnition  The set of functions{xl_{(t) =}_xl_(t,_πl_),_t_∈_[t

l–,tl],l= ,r}is said to be a

so-lution of equations ()-() with variable structure corresponding to an elementπr∈Ar

if the functionxl_(t)_∈_O

l on the interval [tl–,tl] satisﬁes condition () at pointtl, is

ab-solutely continuous on the interval [tl–,tl] with probability , and satisﬁes equation ()

almost everywhere.

Deﬁnition  The elementπr∈Ar is said to be admissible if the pairs (xl(t),ul(t)),t∈

[tl–,tl],l= ,r, are solutions of system ()-() and satisfy the constraints ().

Deﬁnition  LetA

r be the set of admissible elements. The elementπ¯r∈Ar is said to be

an optimal solution of problem ()-() if there exist admissible controlsu¯l_(t),_t_∈_[t l–,tl],

l= ,r, and the corresponding solutionsx¯l_(t),_t_∈_[t

l–,tl],l= ,r, of system ()-() such

that the pairs (¯xl(t),u¯l(t)),l= ,r, minimize the functional ().

3 Maximum principle

The following is a revised and improved version of Theorem  in [].

Theorem  Suppose that,conditions(AI)-(AIII)hold and

πr=t,t,tr,x(t),x(t), . . . ,xr(t),u,u, . . . ,ur

(4)

(a) there exist random processes(ψl_(t),_βl_(t))_∈_L

F(tl–,tl;Rnl)×LF(tl–,tl;Rnl×nl)that

are the solutions of the following adjoint equations:

⎧ ⎪ ⎨ ⎪ ⎩

dψl_{(t) = –H}l

x(ψl(t),xl(t),ul(t),t)dt+βl(t)dwl(t), tl–≤t<tl,

ψl(tl) = –ϕlx(xl(tl)) +ψl+(tl)lx(xl(tl),tl), l= ,r– ,

ψr(tr) = –ϕxr(xr(tr));

()

(b) ∀¯ul∈Ul,l= ,r,a.c.fulﬁlls the maximum principle:

Hlψl(θ),xl(θ),u¯l,θ–Hlψl(θ),xl(θ),ul(θ),θ≤, a.e.θ∈[tl–,tl]; ()

(c) the following transversality conditions hold:

ψl+(tl)lt

xl(tl),tl

= , a.c.,l= ,r– , ()

where,fort∈[tl–,tl],

Hlψ(t),x(t),u(t),t=ψ(t)glx(t),u(t),t+β(t)flx(t),t–plx(t),u(t),t.

The following result can be proved according to the scheme in the proof of Theorem  in []. In order to investigate the optimal control problem with constraints (), we use Ekeland’s variational principle.

Theorem  Suppose that conditions(AI)-(AIV)hold and

πr=t,t,tr,x(t),x(t), . . . ,xr(t),u,u, . . . ,ur

is an optimal solution of problem()-().Then (a) there exist random processes(ψl_(t),_βl_(t))_∈_L

F(tl–,tl;Rnl)×LF(tl–,tl;Rnl×nl)that

are solutions of the following adjoint equations:

⎧ ⎪ ⎨ ⎪ ⎩

dψl(t) = –H_xl(ψl(t),xl(t),ul(t),t)dt+βl(t)dwl(t), tl–≤t<tl,

ψl_(t

l) = –λlqlx(xl(tl)) –λlϕxl(xl(tl)) +ψl+(tl)lx(xl(tl),tl), l= ,r– ,

ψr_(t

r) = –λrqxr(xr(tr)) –λrϕxr(xr(tr));

()

(b) ∀¯ul_∈_Ul_,_l_{= ,}_r_,_a_._c_._{fulﬁlls the maximum principle}_:

Hlψl(θ),xl(θ),u¯l,θ–Hlψl(θ),xl(θ),ul(θ),θ≤, a.e.θ∈[tl–,tl]; ()

(c) the following transversality conditions hold:

ψl+(tl)lt

xl(tl),tl

= , a.c.,l= ,r– , ()

where,fort∈[tl–,tl],

(5)

4 The second-order necessary condition

In order to review in detail the stochastic singular control switching systems, we need to introduce the deﬁnition of singular controls in the maximum principle sense.

Deﬁnition  Admissible controlsul(t),l= , . . . ,r, are said to be singular on control re-gionsVlif eachVl⊂Ulis nonempty and, fora.e. t∈[tl–,tl), we have

H_xlψl(t),xl(t),ul(t),t=H_xlψl(t),xl(t),vl,t, ∀vl∈Vl.

To state the main result of this paper, we will use the following theorem, the full proof of which can be found in []. However, for convenience of the reader, we present a brief proof.

Theorem  Suppose that conditions(AI)-(AIII)and(BI)-(BIII)hold.Letπr= (t,t, . . . ,

tr,x(t),x(t), . . . ,x(tr),u,u, . . . ,ur) be an optimal solution of problem ()-(), and u=

(u,u, . . . ,ur)be singular on the control regionV= (V,V, . . . ,Vr).Then

(a) there exist random processes(ψl(t),βl(t))∈L_Fl(tl–,tl;Rnl)×L_Fl(tl–,tl;Rnl×nl)and

(l_{(t), K}l_(t))_∈_L

Fl(tl–,tl;Rnl)×L_Fl(tl–,tl;Rnl×nl)that are solutions of the following

adjoint equations:

⎧ ⎪ ⎨ ⎪ ⎩

dψl(t) = –H_xl(ψl(t),xl(t),ul(t),t)dt+βl(t)dw(t), tl–≤t<tl,

ψl_(t

l) = –ϕlx(xl(tl)) +ψl+(tl)lx(xl(tl),tl), l= ,r– ,

ψr_(t

r) = –ϕxr(xr(tr));

()

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

dl(t) = –[Hl_x(_tl,xl(t),ul(t),t) +H_xxl (ψl(t),xl(t),ul(t),t) +f_xl∗(xl_(t),_t)l_(t)fl

x(xl(t),t)]dt+ Kl(t)dwl(t), t∈[tl–,tl),

l_(t

l) = –ϕxxl (xl(tl)) +l+(tl)lxx(xl(tl),tl), l= ,r– ,

r(tr) = –ϕrxx(xr(tr));

()

(b) for a.e.θ∈[tl–,tl]and∀¯ul∈Vl,l= ,r,a.c.the second-order maximum conditions

fulﬁll

_u_¯lH_xl

ψl(θ),xl(θ),ul,θ_u_¯lgl∗

xl(θ),ul(θ),θ +_u_¯lgl∗

xl(θ),ul(θ),θl(θ)_u_¯lgl

xl(θ),ul(θ),θ≤; () (c) the following transversality conditions hold:

ψl+(tl)lt

xl(tl),tl

+ .l+∗(tl)ltt

xl(tl),tl

l+(tl) = ,

l= ,r– ,a.c. ()

Here

Hl(t),x(t),u(t),t=(t)glx(t),u(t),t+gl∗x(t),u(t),t(t) + K(t)flx(t),t+fl∗x(t),tK(t).

Proof Letu¯l_{(t) =}_ul_{(t) +}_u_¯l_(t),_l_{= ,}_{r, be some admissible controls and}_x_¯l_{(t) =}_xl_{(t) +}

x¯l_(t),_l_{= ,}_{r, be the corresponding trajectories of system ()-(). Let  =}_t

(6)

tr=Tbe a switching sequence. Then we obtain the following identities for some sequence

 =¯t<t¯<· · ·<¯tr=T:

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

dx¯l_{(t) = [} ¯

ulgl(xl(t),ul(t),t) +g_xl(xl(t),ul(t),t)x¯l(t)

+ .x∗(t)gl

xx(xl(t),ul(t),t)x(t)]dt

+ [fl

x(xl(t),t)x¯l(t) + .x∗(t)fxxl(xl(t),t)x(t)]dwl(t) +ηt,

x¯(t) = ,

x¯l_(t

l–) =l–(¯xl–(tl–),t¯l–) –l–(xl–(tl–),tl–), l= ,r,

()

where

η_t=

 

g_xl∗xl(t) +μx¯l(t),u¯l_t,t–gl∗_xxl(t),ul(t),tx¯l(t)dμdt

+ .

 

x¯l∗(t)g_xxl∗xl(t) +μx¯l(t),ul(t),t–g∗_xxxl(t),ul(t),tx¯l(t)dμdt

+

 

f_xl∗xl_{(t) +}_μ_x_¯l_(t),_t_–_fl∗ x

xl_(t),_t_x_¯l_(t)_dμ_dwl_(t)

+ .

 

x¯l∗(t)f_xxl∗xl(t) +μx¯l(t),t–f_xx∗xl(t),tx¯l(t)dμdwl(t).

In order to establish the existence and uniqueness of a solution of adjoint stochastic diﬀerential equations, it suﬃces to follow the method described in the article [] and to use the independence of Wiener processesw_{(t), . . . ,}_wr_(t).

The stochastic processesψl(t),l= ,r, at the pointst,t, . . . ,trare deﬁned as

ψl(tl) = –ϕlx

xl(tl)

+ψl+(tl)lx

xl(tl),tl

, ψr(tr) = –ϕxr

x(tr)r

()

and

l(tl) = –ϕxxl

xl(tl)

+l+(tl)lxx

xl(tl),tl

, r(tr) = –ϕxxr

xr(tr)

. ()

According to Itô’s formula [], the expression of an increment of the cost functional () takes the following form:

J(u) = r l= E

ϕlx¯l(tl)

–ϕlxl(tl)

+

tl

tl–

plx¯l(t),u¯l_t,t–plxl(t),ul(t),tdt

= – r l= E tl t–

_u_¯lHl

ψl(t),xl(t),ul(t),t+H_xlψl(t),xl(t),ul(t),t¯xl(t)

– .x¯l∗(t)f_xl∗xl(t),tl(t)f_xlxl(t),tx¯l(t) +x¯l∗(t)_u_¯lgl

xl(t),ul(t),tl(t)x¯l(t)

–¯xl∗(t)g_xlxl(t),ul(t),tl(t)¯xl(t) –¯xl∗(t)f_xlxl(t),tK_tl¯xl(t) +ψ_tl∗_u_¯lg_xl

xl(t),ul(t),tx¯l(t) –_u_¯lpl_x

xl(t),ul(t),tx¯l(t)¯tldt

+

r–

l=

ψl+(tl)t

xl(tl),tl

+

r

l=

ηtl

(7)

where

ηtl

tl–=E

 

( –μ)ϕ_xl∗x¯l(tl)

–ϕ_x∗xl(tl)

x¯l(tl)dμ

–E

tl

tl–

 

( –μ)H_xlψl(t),x¯l(tl),ul(t),t

–H_xlψl(t),xl(t)ul(t),t¯xl(t)¯tldμdt

–E

 

( –μ)ψl+ tl

l_xx¯l(tl)

–l x

xl(t),tl

xl(t)¯tldμ

+E

 

( –μ)x¯l∗_t_lϕ_xxl∗x¯l(tl)

–ϕ∗_xxxl(tl)

x¯l(tl)dμ

+E

tl

tl–

 

( –μ)x¯l∗_t H_xxl ψl(t),x¯l(tl),ul(t),t

–H_xxl ψl(t),xl(t)ul(t),tx¯l(t)¯tldμdt

+E

 

( –μ)x¯l∗_t

l

l+_(t

l)

l_xxx¯l(tl),tl

–l_xxxl(t),tl

x¯l(tl)¯tldμ. ()

By assumptions (AIII) and (BIII), using identities () and () from expressions (), (), and (), according to the fact that the coeﬃcients of the independent incrementsxl(t), ¯tequal zero, we obtain that () is true. Taking into consideration (), (), (), and (), through the simple expression () of transformations, can be written as

J(u) =

s

l=

Jlul

= – s l= E tl

tl–

_ulHl

ψl(t),xl(t),ul(t),t+_ulHl

xl

ψl(t),xl(t),ul(t),t¯xl(t)

– .x¯l∗_t_lfl∗xl(t),tl(t)f_xlxl(t),tx¯l(tl)

+x¯l∗(tl)u¯lgl

xl(t),ul(t),tl(t)x¯l(tl)

–x¯l∗(tl)gxl

xl(t),ul(t),tl(t)x¯l(tl) –x¯l∗(tl)fxl

xl(t),tKl(t)x¯l(tl)

_¯

tldt

+

r

l=

ηtl

tl–. ()

Consider the following spike variations:

ul(t) =uθl

t,εl=

, t∈/[θl,θl+εl),εl> ,θl∈[tl–,tl),

¯ ul_–_ul

t, t∈[θl,θl+εl),u¯l∈L(,Fθl,P;Rm),

whereεl>  are small enough.

The following lemma will be used in estimation of increment ().

Lemma  Assume that conditions(AI), (AII), (BI), (BII)are satisﬁed.Then

lim

εl→

Exθl

εl(t) –x

l_(t)_≤

(8)

Here xθl

εl(t) are the trajectories of system()-() corresponding to the controls u

θl

εl(t) =

ul_{(t) +}_uθl

εl(t).

Proof See Lemma  in [].

By invoking expression () and using Lemma  the following estimation is implied:

ηθl+εl

θl =o

ε_l, l= ,r.

According to the singularity of controlsu¯l_(t),_l_{= ,}_{r, in the maximum sense for given}

special variation from (), it follows that, for eachl,

_θlJ(u) = –εlE

_u_¯lHx

ψl(θl),xl(θl),ul(θl),θl

_u_¯lgl

xl(θl),ul(θl),θl

+_u_¯lgl∗

xl(θl),ul(θl),θl

l(θl)u¯lgl

xlθ_l,ul(θl),θl

_¯

tl

+oε_l≥.

Finally, due to the smallness and arbitrariness of εl, () is fulﬁlled. Theorem  is

proved.

5 Necessary condition of optimality for singular controls

This section is devoted to investigation of singular optimal control problems for stochas-tic switching systems with constraints. We obtain a necessary condition of optimality for stochastic control problem of switching systems ()-(). Using Ekeland’s variational prin-ciple, the given problem is converted into a sequence of unconstrained problems. Based on Theorem , we establish the maximum principle and transversality conditions for the sequence of switching systems. Then we obtain a necessary condition of optimality in the case where endpoint constraints are imposed.

Theorem  Suppose that conditions(AI)-(AIV)and(BI)-(BIV)hold.Letπr_{= (t} ,t, . . . ,

tr,x(t),x(t), . . . ,x(tr),u,u, . . . ,ur) be an optimal solution of problem ()-(), and u=

(u_,_u_{, . . . ,}_ur₎_{be singular on the control region}_V_{= (V}_,_V_{, . . . ,}_Vr_)._Then

(a) there exist random processes(ψl_(t),_βl_(t))_∈_L

Fl(tl–,tl;Rnl)×L_Fl(tl–,tl;Rnl×nl)and

(l(t), Kl(t))∈L_Fl(tl–,tl;Rnl)×L_Fl(tl–,tl;Rnl×nl)that are solutions of the following

adjoint equations:

⎧ ⎪ ⎨ ⎪ ⎩

dψl_{(t) = –H}l

x(ψl(t),xl(t),ul(t),t)dt+βl(t)dw(t), tl–≤t<tl,

ψl(tl) = –λlqlx(xl(tl)) –λlϕxl(xl(tl)) +ψl+(tl)lx(xl(tl),tl), l= ,r– ,

ψr(tr) = –λrqxr(xr(tr)) –λrϕxr(xr(tr));

()

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

dl_{(t) = –[H}l

x(l(t),xl(t),ul(t),t) +Hxxl (ψl(t),xl(t),ul(t),t)

+f_xl∗(xl(t),t)l(t)f_xl(xl(t),t)]dt+ Kl(t)dwl(t), t∈[tl–,tl),

l_(t

l) = –λlqlxx(xl(tl)) –λlϕxxl (xl(tl))

+l+_(t

l)lxx(xl(tl),tl), l= ,r– ,

r(tr) = –λrqrxx(xr(tr)) –λrϕrxx(xr(tr));

()

(b) a.e.θ∈[tl–,tl]and∀¯ul∈Vl,l= ,r,a.c.the second-order maximum conditions()

are satisﬁed;

(9)

Proof For any naturalj, let us introduce the following approximating functional for each l= ,r:

Ij(u) =Sj

E

r

l=

ϕl(tl) +

tl

tl–

plxl(t),ul(t),tdt ,E

r

l=

qlxl(tl)

= min

(c,yl₎_∈_ε

c– /j–EM(x, u, t)+

r

l=

yl_–_Eql_xl_(t

l)



,

whereM(x, u, t) =r_i_=[ϕl_(xl_(t

l)) +

tl

tl–p(x

l_(t),_ul_(t),_t)_dt],_ε₌_{_c_:_c_≤_J_,_yl_∈_Gl_}_,_c₌_c₊

· · ·+cr_{, and}_J_{is the minimal value of the functional in problem ()-().}

LetVl≡(U∂l,d) be the space of controls obtained by means of the following metric: d(ul_,_vl_{) = (l}_⊗_P){(t,_ω)_∈_[t

l–,tl]×:νtl=ul(t)}. For eachl= ,r,Vlis a complete metric

space [].

It is easy to prove the following fact.

Lemma  Assume that conditions(AI)-(AIV)hold,ul,n_(t),_l_{= ,}_r,_{is a sequence of}

admis-sible controls from Vl_, _{and x}l,n_(t)_{is the sequence of corresponding trajectories of system}

()-().If d(ul,n_(t),_ul_(t))_→_,_then,_lim

n→∞{suptl–≤t≤tlE|x

l,n_{(t) –}_xl_(t)_|_}_{= ,}_{where x}l_(t)_is

a trajectory corresponding to the admissible controls ul(t),l= ,r.

Due to the continuity of the functionalsI_jl:Vl_→_Rnl_{, according to Ekeland’s variational}

principle, there are controls such thatul,j_{(t) :}_d(ul,j_(t),_ul_(t))_≤_εl

jand for∀ul(t)∈Vl, the

following is achieved:I_jl(ul,j₎_≤_Il

j(ul) +

ε_jld(ul,j_,_ul_),_εl

j=j.

This inequality means that (t, . . . ,tr,x,j(t), . . . ,xr,j(t),u,j(t), . . . ,ur,j(t)) for eacht∈(tl–,tl]

is a solution of the following problem:

⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩

Jj(u) =

r

l=(Ilj(ul) +

εl_jEtl

tl–δ(u

l_(t),_ul,j_(t))_dt)_→_min_,

dxl_{(t) =}_gl_(xl_(t),_ul_(t),_t)_dt₊_fl_(xl_(t),_t)_dw(t), _l_{= ,}_r,

xl+_(t

l) =l(xl(tl),tl), l= ,r– ;x(t) =x,

ul(t)∈U∂l.

()

The functionδ(u,v) is determined in the following way:

δ(u,v) =

, u=v, , u=v.

Then according to Theorem , we have that if (x,j_{(t), . . . ,}_xr,j_(t),_u,j_{(t), . . . ,}_ur,j_{(t)) is an}

optimal solution of problem (), then

() there exist random processes (ψl,j_(t),_βl,j_(t))_∈_L

Fl(tl–,tl;Rnl)×L_Fl(tl–,tl;Rnl×nl) that

are solutions of the following system:

⎧ ⎪ ⎨ ⎪ ⎩

dψl,j_{(t) = –H}l

x(ψl,j(t),xl,j(t),ul,j(t),t)dt+βl,j(t)dwl(t), t∈[tl–,tl),l= ,r,

ψl,j(tl) = –λl,jϕxl(xl,j(tl)) –λl,jqlx(xl,j(tl)) +ψtll+ l

x(xl,j(tl),tl), l= ,r– ,

ψr_(t

r) = –λr,jϕrx(xr,j(tr) –λr,jqrx(xr,j(tr)),

(10)

and the random processesl,j_(t)_∈_L

Fl(tl–,tl;Rnl), Kl,j(t)∈L_Fl(tl–,tl;Rnl×nl) that are

solu-tions of the following system:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

dl,j_{(t) = –[H}l

x(l,j(t),xl,j(t),ul,j(t),t) +Hxxl (ψl,j(t),xl,j(t),ul,j(t),t)

+f_xl∗(xl,j(t),ul,j(t),t)l,j(t)f_xl(xl,j(t),ul,j(t),t)]dt+ Kl,j(t)dwl(t), l,j_(t

l) = –λl,jϕxxl (xl,j(tl)) –λl,jqlxx(xl,j(tl) +tll,j+ l

xx(xl,j(tl),tl), l= ,r– ,

r,j(tr) = –λ r,j

ϕxxr (xr,j(tr)) –λ r,j

 qrxx(xr,j(tr),

()

where nonzero (λl_,j,λ_l,j)∈Rk+_,_l_{= ,}_{r, satisfy the following requirement:}

λl_,j,λl_,j=

–cl+ /j+ϕlxl(tl)

+

tl

tl–

pxl(t),ul(t),tdt, –yl+Eqlxl,j(tl) Jj ()

with

J_j=

_r

l=

yl–Eqlxl,j(tl)+

c– /j–E

r

l=

ϕlxl(tl)

+

tl

tl–

pxl(t),ul(t),tdt

/

;

() for a.e.t∈[tl–,tl] and∀˜ul∈Vl,l= ,r, a.c. is satisﬁed:

_u_¯l,jH_xl

ψl,j(θ),xl,j(θ),ul,j,θ_u_¯l,jgl∗

xl,j(θ),ul,j(θ),θ +_u_¯l,jgl∗

xl,j(θ),ul,j(θ),θl,j(θ)_u_¯lgl

xl,j(θ),ul,j(θ),θ≤; ()

() the following transversality conditions hold:

ψl+,j(tl)lt,j

xl,j(tl),tl

+ .l+,j∗_(t l)ltt,j

xl,j(tl),tl

l+,j(tl) = , l= ,r– ,a.c. ()

According to conditions (BI)-(BIV), we have that (λj_,λj_)→(λ,λ) asj→ ∞.

The proof of item (a) of Theorem  is based on the following lemmas.

Lemma  Letψl(tl)be a solution of system(),andψl,j(tl)be a solution of system().If

d(ul,j_(t),_ul_(t))_→_,_j_{→ ∞,}_then

E

tl

tl–

ψl,j(t) –ψl(t)dt+E

tl

tl–

βl,j(t) –βl(t)dt→, l= ,r.

Proof It is clear that∀t∈[tl–,tl],

dψl,j(t) –ψl(t)

= –H_xlψ_tl,j,xl,j(t),ul,j(t),t–H_xlψl(t),xl(t),ul(t),tdt+βl,j(t) –βl(t)dw(t) = –ψl,j(t)g_xlxl,j(t),ul,j(t),t+βl,j(t)f_xlxl,j(t),t

–pl_xxl,j(t),ul,j(t),t–ψl(t)g_xlxl(t),ul(t),t

(11)

Let us square both sides of the last equation. Then, according to Itô’s formula, ∀s∈ [tr–,T],

Eψl,j(t) –ψl(t)–Eψl,j(s) –ψl(s) = E

tr

s

ψl,j(t) –ψl(t)g_xl∗xl,j_(t),_ul,j_(t),_t_–_gl∗ x

xl_(t),_ul_(t),_t_ψl,j_(t)

+g_xl∗xl(t),ul(t),tψl,j(t) –ψl(t)+f_xl∗xl,j(t),t–f_xl∗xl(t),tβl,j(t) +f_xl∗xl(t),tβl,j(t) –βl(t)–plxl,j(t),ul,j(t),t+pl_xxl(t),ul(t),tdt +E

tr

s

βl,j(t) –βl(t)dt.

Now, due to assumptions (AI)-(AIII), we get:

E

tr

s

βl,j(t) –βl(t)dt+Eψl,j(s) –ψl(s)

≤EN

tr

s

ψl,j(t) –ψl(t)dt+ENε

tr

s

βl,j(t) –βl(t)dt+Eψl,j(tr) –ψl(tr) 

.

Hence, by the Gronwall inequality [] we obtain that

Eψl,j(s) –ψl(s)≤DeN(tr–s) _{a.e. in [t}

r–,tr], ()

whereD=E|ψl,j_(t

r) –ψl(tr)|. Hence, it follows from () and () thatψl,j(tr)→ψl(tr),

which leads toD→. Consequently, it follows thatψl,j(s)→ψl(s) inL

F(tr–,tr;Rnl), and

thusβl,j(s)→βl(s) inL

F(tr–,tr;Rnl×nl).

Then from the expression

Eψl,j(tl) –ψl(tl)–Eψl,j(s) –ψsl 

= E

tl

s

ψl,j(t) –ψl(t)g_xl∗xl,j(t),ul,j(t),t–g_xl∗xl(t),ul(t),tψl,j(t) +g_xl∗xl(t),ul(t),tψl,j(t) –ψl(t)+f_xl∗xl,j(t),t–f_xl∗xl(t),tβ_tj +f_xl∗xl(t),tβl,j(t) –βl,j(t)+pl_xxl(t),ul(t),t–pl_xxl,j(t),ul,j(t),tdt +E

tl

s

βl,j(t) –βl(t)dt,

using simple transformations, in view of assumptions (AI)-(AIV), we obtain:

E

tl

s

βl,j(t) –βl(t)dt+Eψl,j(s) –ψ_sl

≤EN

tl

s

ψl,j(t) –ψl(t)dt+ENε

tl

s

βl,j(t) –βl(t)dt+Eψl,j(tl) –ψl(tl) 

.

Hence, according to the Gronwall inequality, we have:

Eψl,j(s) –ψl(s)≤DeN(tl–s) _{a.e. in [t}

(12)

where the constantDis determined asD=E|ψl,j_(t

l) –ψl(tl)|, andD→. Then from

() it follows thatψl,j(s)→ψl(s) inL

Fl(tl–,tl;Rn) andβl,j(s)→βl(s) inL_Fl(tl–,tl;Rn×n).

Lemma  is proved.

Lemma  Letl,j_(t

l)be a solution of system(),andl(tl)be a solution of system().

Then

E

tl

tl–

l,j(t) –l(t)dt+E

tl

tl–

Kl,j(t) – Kl(t)dt→, l= ,r,as j→ ∞.

Proof Due to Itô’s formula, from expressions () and () for alls∈[tl–,tl) we have:

dl,j(t) –l(t)= –g_xl∗xl,j(t),ul,j(t),tl,j(t) –g_xl∗xj_t,uj_t,tl(t) +_tl,jg_xlxl,j(t),ul,j(t),t–l(t)g_xlxjt,u

j t,t

+f_xl∗xl,j(t),tKl,j(t) –f_xl∗xl(t),tKl(t)

+H_xxl ψl,j(t),xl,j(t),ul,j(t),t–H_xxl ψl(t),xl(t),ul(t),t dt +Kl,j(t) – Kl(t)dwl(t).

Based on Itô’s formula, from the previous expression we get:

El,j(tl) –l(tl)–El,j(s) –l(s)

≤E

tl

s

l,j(t) –_tlg_xl∗xl,j(t),ul,j(t),t–g_xl∗xjt,u j t,t

l,j(t)

+g_xl,∗xj_t,u_tj,tl,j(t) –l(t)+f_xl∗xl,j(t),t–f_xl∗xl(t),tKl,j(t) +H_xxl ψl,j(t),xl,j(t),ul,j(t),t–H_xxl ψ_tj,xl(t),ul(t),t

+H_xxl ψl,j(t),xl(t),ul(t),t–H_xxl ψl(t),xl(t),ul(t),tdt +E

tl

s

Kl,j(t) – Kl(t)dt.

Then by simple transformations we obtain:

E

tl

s

Kl,j(t) – Kl(t)dt+El,j(t) –l(t)

≤EN

tl

s

l,j(t) –l(t)dt+ENε

t s

Kl,j(t) – Kl(t)dt+El,j(tl) –l(tl) 

.

According to the Gronwall inequality, a.e. in [tl–,tl) we have:

El,j(s) –l(s)≤De–N(tl–s)_,

where the constantDis deﬁned as

D=El,j(tl) –l(tl)+ENε

tl

s

(13)

so thatl,j_(t

r)→l(tr), and hence, according to assumptions (BI)-(BIV) and expressions

() and (), we obtainl,j_(t)_→l_{(t) in}_L

Fr(tr–,tr;Rn) asj→ ∞.

Then, according to suﬃcient smallness of ε, it follows that D→. Consequently, l,j(t)→l(t) inL_Fl(tl–,tl;Rn) and Kl,j(t)→Kl(t) inL_Fl(tl–,tl;Rn×n),l= ,r– . Lemma 

is proved.

Based on Lemma  and Lemma , we can pass to the limit in systems () and (), and the fulﬁlment of () and () is derived. Following a similar scheme and taking the limits in () and (), we can obtain the fulﬁlment of second-order necessary conditions and transversality conditions. Theorem  is proved.

6 Conclusions

This paper is motivated by the increased interest of the research on switching systems. Many theoretical and numerical advances have recently been realized in the ﬁeld of stochastic optimization. The stochastic control problem of switching systems in which the endpoint restrictions are deﬁned with the help of convex closed sets is considered. The results developed in this study can be viewed as stochastic analogues of the problems formulated in [, ] and an extension of the results for switching systems given in [, ]. The main result, Theorem , is a natural evolution of Theorem  in [].

Competing interests

The author declares that she has no competing interests.

Acknowledgements

This work was supported by Scientiﬁc Research Project No. 150F202 of Anadolu University, Turkey.

Received: 4 July 2015 Accepted: 13 December 2015 References

1. Arnold, L: Stochastic Diﬀerential Equations: Theory and Applications. Wiley, New York (1974)

2. Kolmanovsky, VB, Myshkis, AD: Applied Theory of Functional Diﬀerential Equations. Kluwer Academic, Dordrecht (1992)

3. Antsaklis, PJ, Nerode, A: Hybrid control systems: an introductory discussion to the special issue. IEEE Trans. Autom. Control43(4), 457-460 (1998)

4. Boukas, E: Stochastic Switching Systems: Analysis and Design. Birkhäuser Boston, Boston (2006) 5. Liberzon, D: Switching in Systems and Control. Birkhäuser Boston, Boston (2003)

6. Baklacioglu, T, Turan, O, Aydin, H: Dynamic modeling of exergy eﬃciency of turboprop engine components using hybrid genetic algorithm-artiﬁcial neural networks. Energy86, 709-721 (2015)

7. Bengea, SC, Raymond, AC: Optimal control of switching systems. Automatica41, 11-27 (2005)

8. Capuzzo, DI, Evans, LC: Optimal switching for ordinary diﬀerential equations. SIAM J. Control Optim.22, 143-161 (1984)

9. Seidmann, TI: Optimal control for switching systems. In: Proceedings of the 21st Annual Conference on Information Science and Systems, pp. 485-489 (1987)

10. Xu, X, Antsaklis, PJ: Results and perspectives on computational methods for optimal control of switched systems. In: Maler, O, Pnueli, A (eds.) Hybrid Systems: Computation and Control. Lecture Notes in Computer Science, vol. 2623, pp. 540-556 (2003)

11. Afanasyev, VN, Kolmanovsky, VB, Nosov, VP: Mathematical Theory for Synthesizing Control Systems. Higher School, Moscow (2003)

12. Gikhman, II, Skorokhod, AV: Controlled Stochastic Processes. Springer, New York (1979) 13. Fleming, WH, Rishel, RW: Deterministic and Stochastic Optimal Control. Springer, Berlin (1975)

14. Yong, J, Zhou, XY: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999) 15. Kushner, HJ: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control10,

550-565 (1976)

16. Arkin, VI, Saksonov, MT: The necessary conditions of optimality in control problems of stochastic diﬀerential equations. Dokl. Akad. Nauk SSSR244(4), 11-16 (1979)

17. Bensoussan, A: Lectures on Stochastic Control in Nonlinear Filtering and Stochastic Control. Lectures Notes in Mathematics. Springer, Berlin (1982)

18. Bismut, JM: Linear quadratic optimal stochastic control with random coeﬃcients. SIAM J. Control14, 419-444 (1976) 19. Haussman, UG: General necessary conditions for optimal control of stochastic systems. Math. Program. Stud.6, 30-48

(14)

20. Bismut, JM: An introductory approach to duality in optimal stochastic control. SIAM Rev.20, 62-78 (1978) 21. Borkar, V: Controlled diﬀusion processes. Probab. Surv.2(4), 213-244 (2005)

22. Makhmudov, NI: General necessary optimality conditions for stochastic systems with controllable diﬀusion. In: Statistics and Control of Stochastic Processes, pp. 135-138. Nauka, Moscow (1989)

23. Peng, S: A general stochastic maximum principle for optimal control problem. SIAM J. Control Optim.28, 966-979 (1990)

24. Aghayeva, CA, Abushov, QU: The maximum principle for the nonlinear stochastic optimal control problem of switching systems. J. Glob. Optim.56(2), 341-352 (2013). doi:10.1007/s10898-011-9825-8

25. Abushov, Q, Aghayeva, C: Stochastic maximum principle for the nonlinear optimal control problem of switching systems. J. Comput. Appl. Math.259, 371-376 (2014). doi:10.1016/j.cam.2013.06.010

26. Aghayeva, C, Abushov, Q: The maximum principle for some nonlinear stochastic optimal control problem of switching system. Trans. Acad. Sci. Azerb. Ser. Phys.-Tech. Math. Sci.31(3), 74-81 (2011)

27. Aghayeva, C, Abushov, Q: Stochastic maximum principle for switching systems. In: Proceedings of the 4th International Conference ‘Problem of Cybernetics and Informatics’, vol. 3, pp. 198-201 (2012)

28. Aghayeva, C, Abushov, Q: Stochastic optimal control problem for switching system with controlled diﬀusion coeﬃcients. In: Proceedings Book of the World Congress on Engineering, vol. I, pp. 202-207 (2013) 29. Gabasov, R, Kirillova, FM: Singular Optimal Control. Plenum, New York (1982)

30. Mordukhovich, B: Approximation Methods in Problems of Optimization and Control. Nauka, Moscow (1988) 31. Makhmudov, NI, Bashirov, AE: First order and second order necessary conditions of optimality for stochastic systems.

In: Proceedings of Steklov Mathematical Institute Seminar, Statistics and Control of Stochastic Processes, pp. 283-295 (1997)

32. Tang, S: A second-order maximum principle for singular optimal stochastic control. Discrete Contin. Dyn. Syst., Ser. B

14(4), 1581-1599 (2010)

33. Aghayeva, C, Alpaslan, M: Singular optimal control problem of stochastic switching systems. In: Proceedings of the World Congress on Engineering and Computer Science, vol. II, pp. 855-859 (2014)