Stochastic linear quadratic control problem of switching systems with constraints

(1)

R E S E A R C H

Open Access

Stochastic linear quadratic control

problem of switching systems with

constraints

Charkaz Aghayeva

*

*_{Correspondence:}

[email protected] Department of Industrial Engineering, Anadolu University, Eskisehir, Turkey

Institute of Control Systems, ANAS, Baku, Azerbaijan

Abstract

This paper is devoted to the optimal control problem for stochastic linear switching systems with a quadratic cost functional. A necessary and suﬃcient condition of optimality for mentioned linear control systems under endpoint constraints is obtained. A linear quadratic controller is simply constructed via a set of stochastic backward Riccati equations.

Keywords: stochastic linear system; conditions of optimality; switching systems; transversality conditions

1 Introduction

The Linear Quadratic (LQ) problem was mathematically formulated and solved, as well as the ﬁltering one, in the s by Kalman []. An important advantage of the LQ theory is the existence of explicit feedback forms for optimal state control and the optimal cost value through the Riccati equations. The deterministic Riccati equation was essentially solved by Wonham [] by applying Bellman’s principle of quasilinearization []. A detailed research of stochastic LQ control problems has been performed by Bismut []. The existence of a unique solution for the associated Riccati equations was studied in [].

Switching systems are more advantageous models to describe the noninvariant phenom-ena with the continuous law of movement and they have gained considerable attention in science and engineering. Examples of these systems include many evolutionary pro-cesses, robotics, integrated circuit design, multimedia, manufacturing, power electronics, chaos generators, and air traﬃc management systems [, ]. Optimization problems have also received growing interest among the researchers of deterministic and for stochastic switching control systems [–].

Manifold problems of stochastic optimal control theory have been considered in [– ]. Optimal control problems of switching systems have attracted considerable attention, due to the advantages, for instance, in modeling and improving the transient response on highly complex systems and systems with large uncertainties. The stochastic maximum principle via backward stochastic diﬀerential equations is derived in [–]. The neces-sary conditions of optimality for stochastic switching systems earlier have been obtained in [–]. In [] the linear quadratic control problem has been investigated for a special type of stochastic systems.

(2)

In this paper, the LQ problem of stochastic switching systems with restrictions is con-sidered. Ekeland’s variational principle [] has been used to establish the necessary and suﬃcient conditions of optimality for a given problem.

2 Statement of main problem

Unless speciﬁed otherwise, throughout the paper we use the same notations as in []. Consider the following stochastic linear control system:

dxl(t) =Al(t)xl(t) +Bl(t)ul(t) +gl(t)dt

+Cl(t)xl(t) +Dl(t)ul(t) +fl(t)dwl(t), t∈(tl–,tl], ()

xl(tl–) =l–(tl–)xl–(tl–) +Kl–(tl–), l= ,r;x(t) =x, () ul(t)∈U∂l ≡

ul(·,·)∈L_Fl|ul(t,·)∈Ul⊂Rml, a.c.

. ()

The elements ofU∂l, are called admissible controls.

Our goal is to ﬁnd an optimal solution (x,u) = (x_,_x_{, . . . ,}_xr_,_u_,_u_{, . . . ,}_ur_{) and a switching}

sequence t = (t,t, . . . ,tr), that minimize the cost functional:

J(u) =E

r

l=

Glxl(tl),xl(tl) +

tl

tl–

Ml(t)xl(t),xl(t) +Nl(t)ul(t),ul(t) dt

, ()

on the decisions of the system ()-() under the conditions:

Eql,xl(tl) ∈Ql, l= , . . . ,r, ()

whereQ_{, . . . ,}_Qr_{are a closed convex sets in}_R_{. The elements of matrices}_Al_,_Bl_,_Cl_,_Dl_,l_,

Ml_,_Nl_{and vectors}_Gl_,_Kl_,_gl_,_fl_{are continuous, bounded functions.}_Gl_,_Ml_{are a positively}

semi-deﬁned matrices, andNl_{are positively deﬁned matrices.}

Airepresents the set of elementsπi= (t,t,ti,x(t),x(t), . . . ,xi(t),u,u, . . . ,ui) for each

i= , . . . ,r. To describe the main result we need to introduce some concepts, such as a solution of linear switching systems, admissible element of control problem and optimal solution for LQ problem of stochastic switching systems. For a detailed account we refer the reader to [, ].

3 Stochastic LQ problem of switching systems

This section is devoted to the investigation of optimal control problems for linear stochas-tic switching systems with constraints. The LQ problem belongs to a special class of con-vex control problems for which the maximum principle is a necessary as well as suﬃcient condition of optimality. The next theorem provides necessary and suﬃcient conditions of the optimality of stochastic linear switching systems.

Theorem  Let A_r be a set of admissible elements.The elements

πr=t,t,tr,x(t),x(t), . . . ,xr(t),u,u, . . . ,ur

∈A_r

(3)

(a) there exist random processes(ψl₍_t_),_βl₍_t₎₎_∈_L

F(tl–,tl;Rnl)×LF(tl–,tl;Rnl×nl)which

are the solutions of the following stochastic backward equations:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

dψl(t) = –[Al∗₍_t₎_ψl₍_t_{) +}_Cl∗₍_t₎_βl₍_t_{) –}_Ml₍_t₎_x₍_t_)]

+βl(t)dwl₍_t_), _t

l–≤t<tl,

ψl(tl) = –λlGlxl(tl) –λlql+ψl+(tl)l(tl), l= , . . . ,r– ,

ψr₍_t

r) = –λrGrxr(tr) –λrqr;

()

(b) the candidate optimal controlsul_∈_Ul_,_l_{= ,}_r_,_{are deﬁned by}

Nl∗(t)ul(t) =Bl∗(t)ψl(t) +Dl∗(t)βl(t), a.e.θ∈[tl–,tl]; ()

(c) the following transversality conditions hold:

ψl+(tl)

l_t∗tlxl(tl) +Ktl∗(tl)

= , a.c.,l= , . . . ,r– . ()

Proof First we investigate given optimal control problem without endpoint constraints (). Letul₍_t_{) and}_u_¯l₍_t_),_l_{= , . . . ,}_r_{be some admissible controls and}_xl₍_t_),_x_¯l₍_t_{) be corresponding}

trajectories. u¯l₍_t_{) represents the admissible increment of the control}_ul₍_t_{). t = (}_t

,t, . . .tr)

andt¯= (t¯,t¯, . . . ,t¯r) denote diﬀerent switching laws. The increment of the cost functional

() along the admissible controlu¯= (u¯₍_t_),_u_¯₍_t_{), . . . ,}_u_¯r₍_t_{)) looks like}

J(u),u¯– u =E

r

l=

Glxl(tl),x¯l(tl) –xl(tl)

+ tl

tl–

Ml(t)xl(t),x¯l(t) –xl(t) +Nl(t)ul(t),u¯l(t) –ul(t) dt

. ()

By ()-() the increments of the trajectories are deﬁned as

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

d(x¯l(t) –xl(t))

= [Al₍_t₎₍_x_¯l₍_t_{) –}_xl₍_t_{)) +}_Bl₍_t₎₍_u_¯l₍_t_{) –}_ul₍_t_))]_dt

+ [Cl₍_t₎₍_x_¯l₍_t_{) –}_xl₍_t_{)) +}_Bl₍_t₎₍_x_¯l₍_t_{) –}_xl₍_t_))]_dwl₍_t_), _t_∈₍_t¯ l–,t¯l],

xl+₍_t

l) =x¯l+(t¯l) –xl+(tl) =l(t¯l)x¯l(t¯l) –l(tl)xl(tl).

()

Let us introduce the stochastic processesψl₍_t_),_l_{= , . . . ,}_r_{, as the solution of the following}

stochastic backward diﬀerential equations: ⎧

⎪ ⎨ ⎪ ⎩

dψl₍_t_{) = –[}_Al∗₍_t₎_ψl₍_t_{) +}_Cl∗₍_t₎_βl₍_t_{) –}_Ml₍_t₎_x₍_t_{)] +}_βl₍_t₎_dwl₍_t_), _t

l–≤t<tl,

ψl(tl) = –Glxl(tl) +ψl+(tl)l(tl), l= , . . . ,r– ,

ψr(tr) = –Grxr(tr).

()

According to the Ito formula for eachl= , . . . ,rthe following identity is satisﬁed:

dψl(tl), (x¯l(tl) –xl(tl)( tl

=dψl(t),x¯l(t) –xl(t) tl +

ψl(t),dx¯l(t) –xl(t) tl

+βl(t),Cl(t)x¯l(t) –xl(t) tl+Dl(t)

¯

(4)

Integrating the aforementioned equality and taking the expectation of both sides into account in () it follows

Eψl(tl), (x¯l(tl) –xl(tl)( tl –

ψl(tl–),

¯

xl(tl–) –xl(tl–)

tl–

=E

tl

tl–

dψl(t) +Al∗(t)ψl(t) +Cl∗(t)βl(t),x¯l(t) –xl(t) tl

+E

tl

tl–

Bl∗(t)ψl(t) +Dl∗(t)βl(t),u¯l(t) –ul(t) tl dt.

Due to this equality equation () can be rewritten as

J(u),u¯– u =E

r

l=

Glxl(tl),x¯l(tl) –xl(tl) +

ψl(tl),x¯l(tl) –xl(tl)

–E

tl

tl–

dψl(t) +Al∗(t)ψl(t) +Cl∗(t)βl(t),x¯l(t) –xl(t) tl

–E

tl

tl–

Bl∗(t)ψl(t) +Dl∗(t)βl(t),u¯l(t) –ul(t) tl dt

+ tl

tl–

Ml(t)xl(t),x¯l(t) –xl(t) +Nl(t)ul(t),u¯l(t) –ul(t) dt

. ()

Further, using equation () we get a more succinct expression:

J(u),u¯– u =E

r

l=

tl

tl–

Nl(t)ul(t) –Bl∗(t)ψl(t) –Dl∗(t)βl(t),u¯l(t) –ul(t) tl dt. ()

It is well known that a necessary and suﬃcient condition of optimality for the convex functional is given byJ(u) = . The validity of () and (), hence the necessary conditions of optimality for the considered unrestricted problem ()-() follows from equations () and (). At last, according to the independence of the increments x¯l₍_t_), _u_¯l₍_t_), _t¯

l,

suﬃciency follows from equation ().

To construct the optimality condition of LQ problem ()-() with the right endpoint con-straints (), the above mentioned problem by using Ekeland’s variational principle [] is converted into a sequence of unconstrained problems. Based on the results already ob-tained for problem ()-(), necessary and suﬃcient conditions for the sequence of switch-ing systems are established.

To apply Ekeland’s variational principle we introduce the following approximating func-tional:

Ij(u) = min

(c,yl₎_∈_ε

r

l=

cl_–_εl

j–ESl(x,u,t)



+

r

l=

yl_–_Eql_xl₍_t l)



.

HereSl₍_x_,_u_,_t_{) =}_Gl_xl₍_t

l),xl(tl) +

tl

tl–(M

l₍_t₎_xl₍_t_),_xl₍_t₎ ₊_Nl₍_t₎_ul₍_t_),_ul₍_t₎₎_dt_;_lim j→∞εlj=

;c=c₊_{· · ·}₊_cr_;_ε₌_{_c_:_c_≤_J_,_yl_∈_Ql_}_{; let}_J_{be a minimal value of the functional in the}

(5)

LetVl_≡₍_Ul

∂,d) be the space of controls obtained by means of the following metric:

dul,vl= (l⊗P)(t,ω)∈[tl–,tl]×:νtl=ult

.

For eachl= , . . . ,r, letVl_{be a complete metric space [].}

For the following fact it is signiﬁcant that we can provide a relation between the sequence of controls from the metric spaceVl _{and the sequence of corresponding trajectories of}

system ()-().

Lemma ([], Lemma .) Let(v,n_{, . . . ,}_vr,n₎_{be the sequence of admissible controls from}

(V_{, . . . ,}_Vr_),_and₍_x,n_{, . . . ,}_xr,n₎_{be the sequence of corresponding trajectories of the system}

()-().Let for each l= , . . . ,r the condition d(vl,n_,_ul₎_→__{when n}_{→ ∞}_{be met}_._Then

lim

n→∞

sup

tl–≤t≤tl

Exl,n(t) –xl(t)

= ,

where xl₍_t₎_{is a trajectory corresponding to admissible controls u}l₍_t_),_l_{= , . . . ,}_r_.

Due to Ekeland’s variational principle,we see that(x,j₍_t_{), . . . ,}_xr,j₍_t_),_u,j₍_t_{), . . . ,}_ur,j₍_t₎₎_{is a}

solution of the following problem: ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

Jj(u) =Ij(u) +E

r l=

ε_jltl

tl–δ(u

l₍_t_),_ul,j₍_t₎₎_dt

→mindxl,j₍_t_{) = [}_Al₍_t₎_xl,j₍_t_{) +}_Bl₍_t₎_ul,j₍_t_{) +}_gl₍_t_)]_dt

+ [Cl₍_t₎_xl,j₍_t_{) +}_Dl,j₍_t₎_ul₍_t_)]_dwl₍_t_), _t_∈₍_t l–,tl],

xl,j₍_t

l–) =l–(tl–)xl–,j(tl–) +Kl–(tl–), l= , . . . ,r, x,j(t) =x,

ul,j₍_t₎_∈_Ul

∂.

()

δ(u,v)is the characteristic function of the set{u,v∈Vl_:_u₌_v_}_.

Based on (),it is found that,if (x,j₍_t_{), . . . ,}_xr,j₍_t_),_u,j₍_t_{), . . . ,}_ur,j₍_t₎₎_{is an optimal}

solu-tion of problem (),there exist the random processes(ψl,j(t),βl,j(t))∈L_Fl(tl–,tl;Rnl)×

L

Fl(tl–,tl;Rnl×nl),which are solutions of the following system:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩

dψl,j₍_t_{) = –[}_Al∗₍_t₎_ψl,j₍_t_{) +}_Cl∗₍_t₎_βl,j₍_t_{) –}_Ml₍_t₎_xl,j₍_t_)]

+βl₍_t₎_dwl₍_t_), _t

l–≤t<tl,

ψl,j(tl) = –λl

,j

Glxl,j(tl) –λl

,j

ql+ψl+,j(tl)l(tl), l= , . . . ,r– ,

ψr,j(tr) = –λr,jGrxr,j(tr) –λr,jqr,

()

where the non-zero(λl_,j,λ_l,j),l= , . . . ,r,are deﬁned as

λl_,j,λl_,j=(–c

l₊_εl

j+Sl(xj,uj,t), –yl+Eqlxl,j(tl))

J

j

; ()

here

J_j= _r

l=

yl–Eqlxl,j(tl)

 + r l=

cl–εl_j–ESlxj,uj,t

/

(6)

On the one hand,due to(),∀˜ul_∈_Vl_{the following necessary and suﬃcient condition of}

optimality for the unconstrained problem()holds:

Nl∗(t)ul,j(t) =Bl∗(t)ψl,j(t) +Dl∗(t)βl,j(t), a.e. t∈[tl–,tl],a.c. ()

Besides,based on()and()we see that the optimal decision of the problem()satisﬁes the transversality condition:

ψl+,j(tl)

l_t∗(tl)xl,j(tl) +Ktl∗(tl)

= , a.c.,l= , . . . ,r– . ()

Sincer_l_=|λ_l,j|₊_|_λl,j

|= exists by() (λ

l,j

,λ

l,j

)→(λl,λl)if j→ ∞.

The truth of () is based upon the following lemma, which can be proved by the same method as the proof of Lemma  [].

Lemma  Letψl₍_t

l)be a solution of system(),ψl,j(tl)be a solution of system().If the

sequence of controls(u,j₍_t_{), . . . ,}_ur,j₍_t₎₎_{satisﬁes the assumptions of the Lemma}_,_then

E

tl

tl–

ψl,j(t) –ψl(t)dt+E

tl

tl–

βl,j(t) –βl(t)dt→, l= , . . . ,r.

Based on Lemma , we can pass to the weak limit in system () and obtain the fulﬁllment of (). Following a similar scheme, we take the limits in () and (), and justiﬁcations of (), () are derived. Theorem  is proved.

4 Riccati equations for switching systems

In the theory of LQ problem, it is very natural to connect the LQ problem with the Riccati equation for the possible feedback design. In this section the optimal control is determined explicitly via a set of stochastic Riccati equations. First, we investigate the feedback design problem in the casegl(t)≡;fl(t)≡ and we search a relation in the form:

ψl(t) = –pl(t)xl(t), l= , . . . ,r, a.c. () To determine the stochastic processespl₍_t_{) we introduce the following theorem.}

Theorem  Letψl(t)be a solution of system(),pl₍_t₎_{be a stochastic process that satisﬁes}

(),deﬁned as the solution of the following diﬀerential equation:

dpl(t) = –pl(t)Al(t) +Al∗(t)pl(t) +γl(t)Cl(t) +Cl∗(t)γl(t) +Cl∗pl(t)Cl

+Ml–pl(t)Bl(t) +γl(t)Dl(t) +Cl∗(t)pl(t)Dl(t)Nl(t) +Dl∗(t)pl(t)Dl(t) ×Bl∗(t)pl(t) +Dl∗(t)γl(t) +Dl∗(t)pl(t)Cl(t)

+γl(t)dwl(t), tl–≤t<tl. ()

Proof Suppose that the diﬀerential of random processespl₍_t_{) is deﬁned as}

(7)

According to the Ito formula:

dψl(t) = –dpl(t)xl(t) –pl(t)dxl(t) –γl(t)Cl(t)xl(t) +Dl(t)ul(t)dt, l= , . . . ,r, a.c.

Using () and () we have

–Al∗ψl(t) –Cl∗βl(t) +Ml(t)xl(t)+βl(t)dwl(t) = –αl(t)xl(t)dt+γl(t)xl(t)dwl(t) +pl(t)Al(t)xl(t)dt

+pl(t)Bl(t)ul(t)dt+plCl(t)xl(t)

+Dl(t)ul(t)dwl(t) +γl(t)Cl(t)xl(t) +Dl(t)ul(t)dt, l= , . . . ,r. ()

Taking expectation from both side we obtain the following expression forβl(t),l= , . . . ,r:

βl(t) = –γl(t)xl–pl(t)Cl(t)xl(t) –pl(t)Dl(t)ul(t), t∈[tl–,tl]. ()

By means of simple transformations taking into account () equation () can be rewrit-ten as follows:

αl(t) +pl(t)Al(t) +Al∗(t)pl(t) +γl(t)Cl(t)

+Cl∗(t)γl(t) +Cl∗pl(t)Cl+Mlxl(t)

+pl(t)Bl(t) +γl(t)Dl(t) +Cl∗(t)pl(t)Dl(t)ul(t) = . ()

Considering () in equation () the optimal control can be deﬁned explicitly:

Nl(t) +Dl∗(t)pl(t)Dl(t)ul(t)

+Bl∗(t)pl(t) +Dl∗(t)γl(t) +Dl∗(t)pl(t)Cl(t)xl(t) = . ()

Hence, Theorem  is proved.

Finally, the feedback design for LQ problem ()-() is obtained by means of the next theorem.

Theorem  Letψl₍_t₎_{be a solution of system}_(),_pl₍_t_),_νl₍_t₎_{be a stochastic processes satisfy}

toψl₍_t_{) = –[}_pl₍_t₎_xl₍_t_{) +}_νl₍_t_)],_a_._c_._{Then for each l let the random process}_νl₍_t₎_{be a solution}

of the following diﬀerential equation:

pl(t)Bl(t) +γl(t)Dl(t) +Cl∗pl(t)DlNl(t) +Dl∗(t)pl(t)Dl(t)–Bl∗(t) –Al∗(t)νl(t)dt

+pl(t)Bl(t) +γl(t)Dl(t) +Cl∗pl(t)DlNl(t) +Dl∗(t)pl(t)Dl(t)–Dl∗(t) –Cl∗(t) ×pl∗(t)fl(t) +ϕl(t)dt

=dνl(t) +pl∗(t)gl(t) –γl∗(t)fl(t)dt+ϕl(t)dwl(t), νl(tl) = . ()

(8)

Proof Suppose that random processesνl₍_t_{) are deﬁned in the following way:}

νl(tl) –νl(tl–) =

tl

tl–

κl(t)dt+

tl

tl–

ϕl(t)dwl(t), l= , . . . ,r.

According to Ito’s formula for eachl:

dψl(t) +κl(t)dt+ϕl(t)dwl(t)

= –dpl(t)xl(t) –pl(t)dxl(t) –γl∗(t)Cl(t)xl(t) +Dl(t)ul(t) +fl(t)dt, a.c. ()

In view of () we obtain the following expression:

βl(t) = –γl(t)xl(t) –pl(t)Cl(t)xl(t) +pl(t)Dl(t)ul(t) +pl∗(t)fl(t) +ϕl(t), l= , . . . ,r, a.c.

Substituting this expression forβl₍_t_{) into (), in view of [}_Nl₍_t_{) +}_Dl∗₍_t₎_pl₍_t₎_Dl₍_t_{)] being}

a positively deﬁned matrix, we have

ul(t) = –Nl(t) +Dl∗(t)pl(t)Dl(t)–Bl∗(t)νl(t) +pl(t)Dl(t)fl(t)

+Dl∗(t)ϕl(t) +(pl(t)Bl(t) +Dl∗(t)γl(t) +pl(t)Dl(t)Cl(t)xl(t). ()

Integrating both sides of () and using equation () of ul₍_t_{), bearing in mind that}

stochastic processespl₍_t_),_γl₍_t_{) are the solutions of diﬀerential equation ():}

κl(t) =pl(t)Bl(t) +γl(t)Dl(t) +Cl∗(t)pl(t)Dl(t)Nl(t) +Dl∗(t)pl(t)Dl(t)–

×Bl∗(t) –Al∗(t)νl(t) +pl(t)Bl(t) +γl(t)Dl(t) +Cl∗(t)pl(t)Dl(t)

×Nl(t) +Dl∗(t)pl(t)Dl(t)–Dl∗(t) –Cl∗(t)pl∗(t)fl(t) +ϕl(t)–pl∗(t)gl(t)

–γl∗(t)fl(t). ()

Therefore, the assertion of the theorem is true.

5 Conclusion

There are a lot relevant applications of LQ problems in ﬁelds such as aerospace, biology, economics, management sciences,etc.[–].

Switching systems provide a natural and convenient theoretical account for mathe-matical modeling of many complex real phenomena and practical applications. A broad spectrum of the latest research is concerned with optimal control problems of stochastic switching systems [–].

(9)

Competing interests

The author declares to have no competing interests.

Acknowledgements

The author thanks the two anonymous reviewers whose comments and suggestions helped improve this manuscript. The research underlying this paper is supported by the Scientiﬁc Research Project No. 1505F202 of Anadolu University, Turkey.

Received: 26 January 2016 Accepted: 18 March 2016 References

1. Kalman, RE: Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana5, 102-119 (1960) 2. Wonham, WM: On a matrix Riccati equation of stochastic control. SIAM J. Control Optim.6, 312-326 (1968) 3. Bellman, R: Functional equations in the theory of dynamic programming, positivity and quasilinearity. Proc. Natl.

Acad. Sci. USA41, 743-746 (1955)

4. Bismut, JM: Linear quadratic optimal stochastic control with random coeﬃcients. SIAM J. Control14, 419-444 (1976) 5. Bismut, JM: An introductory approach to duality in optimal stochastic control. SIAM Rev.20, 62-78 (1978) 6. Boukas, E: Stochastic Switching Systems: Analysis and Design. Birkhäuser, Basel (2006)

7. Liberzon, D: Switching in Systems and Control. Birkhäuser, Basel (2003)

8. Bengea, SC, Raymond, AC: Optimal control of switching systems. Automatica41, 11-27 (2005)

9. Capuzzo, DI, Evans, LC: Optimal switching for ordinary diﬀerential equations. SIAM J. Control Optim.22, 143-161 (1984)

10. Seidmann, TI: Optimal control for switching systems. In: Proceedings of the 21st Annual Conference on in Formations Science and Systems, pp. 485-489 (1987)

11. Hoek, J, Elliott, RJ: American option prices in a Markov chain model. Appl. Stoch. Models Bus. Ind.28, 35-39 (2012) 12. Azevedo, N, Pinheiro, D, Weber, G-W: Dynamic programming for a Markov-switching jump-diﬀusion. J. Comput. Appl.

Math.267, 1-19 (2014)

13. Temocin, B, Weber, G-W: Optimal control of stochastic hybrid system with jumps: a numerical approximation. J. Comput. Appl. Math.259, 443-451 (2014)

14. Xu, X, Antsaklis, PJ: Results and perspectives on computational methods for optimal control of switched systems. In: Maler, O, Pnueli, A (eds.) Hybrid Systems: Computation and Control. Lecture Notes in Computer Science, vol. 2623, pp. 540-556 (2003)

15. Haussman, UG: General necessary conditions for optimal control of stochastic systems. In: Stochastic Systems: Modeling, Identiﬁcation and Optimization, II. Mathematical Programming Studies, vol. 6, pp. 30-48 (1976)

16. Kek, SL, Teo, LK, Ismail, AM: An integrated optimal control algorithm for discrete-time nonlinear stochastic system. Int. J. Control83(12), 2536-2545 (2010)

17. Kushner, HJ: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control10, 550-565 (1976)

18. Pham, H: On some recent aspects of stochastic control and their applications. Probab. Surv.2, 506-549 (2005) 19. Hafayed, M, Abbas, S, Abba, A: On mean-ﬁeld partial information maximum principle of optimal control for stochastic

systems with Lévy processes. J. Optim. Theory Appl.167, 1051-1069 (2015)

20. Shen, Y, Siu, TK: The maximum principle for a jump-diﬀusion mean-ﬁeld model and its application to the mean-variance problem. Nonlinear Anal.86, 58-73 (2013)

21. Wang, G, Zhang, C, Zhang, W: Stochastic maximum principle for mean-ﬁeld type optimal control under partial information. IEEE Trans. Autom. Control59(2), 522-528 (2014)

22. Yong, J, Zhou, XY: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999) 23. Borkar, V: Controlled diﬀusion processes. Probab. Surv.2(4), 213-244 (2005)

24. Makhmudov, NI: General necessary optimality conditions for stochastic systems with controllable diﬀusion. In: Statistics and Control of Random Processes, pp. 135-138. Nauka, Moscow (1989)

25. Peng, S: A general stochastic maximum principle for optimal control problem. SIAM J. Control Optim.28, 966-979 (1990)

26. Hafayed, M, Veverka, P, Abbas, S: On near-optimal necessary and suﬃcient conditions for forward-backward stochastic systems with jumps, with applications to ﬁnance. Appl. Math.59(4), 407-440 (2014)

27. Hafayed, M, Abba, A, Boukaf, S: On Zhou’s maximum principle for near-optimal control of mean-ﬁeld

forward-backward stochastic systems with jumps and its applications. Int. J. Model. Identif. Control25(1), 1-16 (2016) 28. Aghayeva, C, Abushov, Q: The maximum principle for the nonlinear stochastic optimal control problem of switching

systems. J. Glob. Optim.56(2), 341-352 (2013)

29. Abushov, Q, Aghayeva, C: Stochastic maximum principle for the nonlinear optimal control problem of switching systems. J. Comput. Appl. Math.259, 371-376 (2014)

30. Aghayeva, C: Necessary conditions of optimality for stochastic switching control systems. Dyn. Syst. Appl.24(3), 243-258 (2015)

31. Aghayeva, C, Abushov, Q: Linear-square stochastic optimal control problem with variable delay on control and state. Transactions ANAS, math.- ph. series, Informatics and Control Problems25(3), 204-208 (2005)

32. Ekeland, I: On the variational principle. J. Math. Anal. Appl.47, 324-353 (1974)

33. Aghayeva, C, Abushov, Q: The maximum principle for some nonlinear stochastic control system with variable structure. Theory Stoch. Process.32(1), 1-11 (2010)

34. Bensoussan, A, Delfour, MC, Mitter, SK: The linear quadratic optimal control problem for inﬁnite dimensional systems over an inﬁnite horizon; survey and examples. In: Proceedings of the IEEE Conference on Decision and Control and the 15th Symposium on Adaptive Processes, pp. 746-751. Inst. Electr. Electron. Engrs., New York (1976)

35. Kohlmann, M, Zhou, X: Relationship between backward stochastic diﬀerential equations and stochastic controls: a linear-quadratic approach. SIAM J. Control Optim.38, 1392-1407 (2000)

(10)

37. Yuan, C, Wu, F: Hybrid control for switched linear systems with average dwell time. IEEE Trans. Autom. Control60(1), 240-245 (2015)

38. Sahin, O: Optimum arrival routes for ﬂight eﬃciency. J. Power Energy Eng.3, 449-452 (2015)

39. Azhmyakov, V, Attia, SA, Raisch, J: On the maximum principle for impulsive hybrid system. Lect. Notes Comput. Sci.

4981, 30-42 (2008)

40. Ghomanjani, F, HadiFarahi, M: Optimal control of switched systems based on Bezier control points. Int. J. Intell. Syst. Appl.7, 16-22 (2012)

41. Wang, S, Wu, Z: Maximum principle for optimal control problems of forward-backward regime-switching systems involving impulse controls. Math. Probl. Eng.2015, 892304 (2015)

42. Debrabant, K, Jakobsen, ER: Semi-Lagrangian schemes for linear and fully nonlinear diﬀusion equations. Math. Comput.82, 1433-1462 (2013)