Outubro, 2008
Nondegenerate forms of the Maximum
Principle for Optimal Control Problems
with State Constraints
Professor Doutor Fernando A. C. C. Fontes
Professora Doutora Maria do Rosário de Pinho
e da
Outubro, 2008
Nondegenerate forms of the Maximum
Principle for Optimal Control Problems
with State Constraints
I would like to express my deep and sincere gratitude to my supervisor Professor Fernando A. C. C. Fontes for his advice, his supervision and his crucial contributions which made him a backbone of this research and thus of this thesis.
Besides I am deeply grateful to Professor Maria do Ros´ario de Pinho, for her detailed and constructive comments, and for her important support throughout this work.
In addition to that I wish to thank Professor Helene Frankowska for giving me the opportunity to work with her and share all her knowledge.
My gratitude also goes to Professor Delfim F. M. Torres and Professor Emmanuel Trelat for their support.
During this work I have collaborated with many colleagues for whom I hold high regard, and I wish to extend my warmest thanks to all those who have helped me with my work in the Department of Mathematics for Science and Technology.
Words fail me to express my appreciation to my husband Emanuel whose love and persistent confidence in me have taken much off my shoulders. His company and support is my source of strength during this strenuous PhD journey. My special gratitude is due to my sister and parents, who have always supported me and have kept me focused.
And last, but definitely not least, I’d like to thank my son and nephew. It is to them that this thesis is dedicated.
The financial support from Project HPMT-CT-2001-00278 of CTS - Control Traning Site, from projecto ”Optimiza¸c˜ao e Controlo” of FCT-Program and from Projecto FCT POSI/EEA-SRI/61831/2004 ”Controlo ´Optimo com Restri¸c˜oes e suas Aplica¸c˜oes” are gratefully acknowledged.
Nondegenerate forms of the Maximum Principle for Optimal
Control Problems with State Constraints
The Maximum Principle (MP) plays an important role in the characterization of solutions to optimal control problems. It typically identifies a small set of candidates where the minimizers belong.
However, for some optimal control problems with constraints, it may happen that the MP is unable to provide any useful information; for example, if the set of candidates to minimizers that satisfy a certain MP coincides with the set of all admissible solutions. When this happens, we say that the degeneracy phenomenon occurs.
One of ours main goals, is preventing the degeneracy phenomenon to occur by imposing additional terms to the MP. In this context, we developed new strength-ened forms of the MP, for optimal control problems and in particular for optimal control problems with higher index state constrains.
Another case where the MP is unable to provide any useful information hap-pens when the scalar multiplier associated with the objective function is equal to zero. So, the MP merely states a relation between the constraints and does not use the objective function to select candidates to minimizers. We have also developed strengthened forms of the MP such that the MP can be written with the multiplier associated with the objective function not zero, the so-called normal forms of the MP, for optimal control problems.
These two types of strengthened forms of the MP can be applied when the prob-iii
lem satisfies additional hypotheses, known as constraint qualifications, and therefore the constraint qualifications are also object of our study.
The nondegenerate forms of MP, that were developed in this thesis, are valid for new types of optimal control problems with state constraints both by addressing problems with less restrictions on its data, and also by developing new constraint qualifications that are verified for more problems or are easier to verify whether they are satisfied.
Formas n˜
ao degeneradas do Princ´ıpio do M´
aximo para
Prob-lems de Control ´
Optimo com Restri¸
c˜
oes de Estado
O Princ´ıpio do M´aximo (PM) tem um papel fundamental na caracteriza¸c˜ao de solu¸c˜oes de problemas de controlo ´optimo. O PM tipicamente identifica um pe-queno conjunto de candidatos entre os quais se encontram o(s) ´optimos.
Contudo, para alguns problemas de controlo ´optimo com restri¸c˜oes, o PM poder´a n˜ao fornecer qualquer informa¸c˜ao ´util; por exemplo, se o conjunto de candidatos a m´ınimos que satisfaz o PM coincide com o conjunto de todas as solu¸c˜oes admiss´ıveis. Quando tal acontece, dizemos que o fen´omeno de degenera¸c˜ao ocorre.
Um dos nossos principais objectivos, ´e garantir a n˜ao ocorrˆencia do fen´omeno de degenera¸c˜ao impondo condi¸c˜oes adicionais ao PM. Neste contexto, desenvolvemos formas fortalecidas do PM para problems de controlo ´optimo e em particular para problemas de controlo ´optimo com restri¸c˜oes de estado de “elevado”´ındice.
Outro caso em que o PM n˜ao fornece informa¸c˜ao ´util, ocorre quando o multi-plicador associado `a fun¸c˜ao objectivo ´e igual a zero. Neste caso o PM ´e uma mera rela¸c˜ao entre as restri¸c˜oes e portanto n˜ao usa a fun¸c˜ao objectivo para seleccionar um conjunto de candidatos a m´ınimos. Desenvolvemos, tamb´em, formas fortalecidas do PM de modo a que possam ser escritas com o multiplicador associado `a fun¸c˜ao ob-jectivo n˜ao nulo, denominadas por PM normais, para problemas de controlo ´optimo. Estes dois tipos de condi¸c˜oes fortalecidas s˜ao aplic´aveis apenas quando o prob-lema satisfaz hip´oteses adicionais, conhecidas como qualifica¸c˜oes de restri¸c˜ao, e por-tanto as qualifica¸c˜oes de restri¸c˜ao s˜ao tamb´em objecto do nosso estudo.
As formas n˜ao degeneradas do PM, desenvolvidas nesta tese, s˜ao v´alidas para novos tipos de problems de controlo ´optimo com restri¸c˜oes de estado, simultanea-mente por permitirem problemas com menos restri¸c˜oes nos dados, e tamb´em por desenvolverem qualifica¸c˜oes de restri¸c˜ao que s˜ao verificadas para um maior n´umero de problemas ou s˜ao mais f´aceis de verificar.
Acknowledgements i
Abstract iii
Resumo v
Notation xi
List of Constraint Qualifications xv
List of Figures xvii
1 Introduction 1
1.1 Scope and Motivation . . . 1
1.2 Overview . . . 2
2 Background 5 2.1 NCO for Calculus of Variations Problems . . . 5
2.2 NCO for Optimal Control Problems . . . 6
2.2.1 The Problems . . . 6
2.2.2 Maximum Principle . . . 8
2.3 Nonsmooth NCO . . . 9
2.3.1 Nonsmooth NCO for Calculus of Variations Problems . . . 10
2.3.2 Nonsmooth NCO for Optimal Control Problems . . . 10 vii
2.3.3 Nonsmooth NCO for Optimal Control Problems with State
Constraints . . . 14
2.4 Existence and Regularity . . . 16
3 The Degeneracy Phenomenon of Necessary Conditions of Optimal-ity 19 3.1 Degeneracy in Mathematical Programming . . . 19
3.2 Degeneracy in Optimal Control Problems . . . 21
3.2.1 Avoiding the Degeneracy Phenomenon . . . 23
4 Normality in Calculus of Variations Problems 35 4.1 Introduction . . . 35
4.2 Normality in CVP Applying the Normal Result of [Fon00] . . . 37
4.3 Normality in CVP Applying the Normal Result of [RV99] or [CF05] . 39 5 Normality of Optimal Control Problems via Linearization of Con-trol Systems 41 5.1 Introduction . . . 41
5.2 Normality Result via Linearization . . . 44
5.3 Proof of Lemmas . . . 46
5.3.1 Proof of Lemma 5.2.3 . . . 46
5.3.2 Proof of Lemmas 5.2.4 . . . 46
5.3.3 Proof of Lemma 5.2.6 . . . 56
6 Nondegeneracy with Integral-type Constraint Qualifications 63 6.1 Introduction . . . 63
6.2 Nondegenerate Maximum Principle with Integral-type CQ . . . 66
6.3 Proof of Theorem 6.2.1 . . . 68
7 Nondegeneracy with easier verifiable Constraint Qualification 79 7.1 Introduction . . . 79
7.3 Proof of Theorem 7.2.1 . . . 84
8 Nondegeneracy with easier verifiable Integral-type Constraint Qual-ification 91 8.1 Easier Verifiable Nondegenerate Result with Integral-type CQ . . . . 92
8.2 Proof of Theorem 8.1.1 . . . 94
9 Nondegeneracy in Problems with Higher Index State Constraints 99 9.1 Introduction . . . 100
9.2 Higher Index . . . 101
9.3 Main Results . . . 105
9.4 Proof of Main Results . . . 107
10 Conclusion 113 10.1 Contributions . . . 113
10.2 Future works . . . 114
Appendix 117
NCO Necessary Conditions of Optimality
CQ Constraint Qualification
OCP Optimal Control Problem
CVP Calculus of Variation Problem
MPP Mathematical Programming Problem
MP Maximum Principle
W1,1(I : R) absolutely continuous functions from I to R
C([I] : R) continuous functions from I to R
C∗([I] : R) dual space of the space of continuous functions C([I] : R)
C1,1 class of functions which are continuously differentiable
with locally Lipschitz continuous derivatives
B closed unit ball in Euclidean space
x + δB ball of radius δ centred at x xi
kxk Euclidean norm of x
kxkX norm of x in the space X
˙x(t) total derivative with respect to t
dom f domain of f
int (C) interior of C
bdy (C) boundary of C
¯
C closure of C
co C convex hull of a set C
dC(x) Euclidean distance of x to the set C
epi f epigraph of f
supp {µ} support of a measure µ
a · b inner product of a and b
(¯x, ¯u) optimal minimizer of a optimal control problem
fx(x) derivative of f with respect to x
C− negative polar cone
CC(x) Clarke’s tangent cone to the set C at x
NC(x) Clarke’s normal cone to the set C at x
NL
C(x) limiting normal cone to the set C at x
∂Lf (x) limiting subdifferential of f at x
˜
∂f (x) Clarke’s subdifferential of f at x
∂>
xh(t, x) hybrid partial subdifferential
Optimal Control Problems
CQ to guarantee Nondegeneracy
CQAA97 CQ from [AA97], . . . 24
CQ1FV94 CQ from [FV94], . . . 25
CQ2FV94 CQ from [FV94](special case - the minimizing trajectory itself leaves the boundary immediately), . . . 25
CQRV00 CQ from [RV00], . . . .27 CQ1d CQ involving ¯u, . . . .32 CQ2d CQ without ¯u, . . . 33 CQFFV99 CQ from [FFV99], . . . 64 CQI Integral type CQ, . . . 64 CQEV Easier verifiable CQ , . . . 82
CQEVI Easier verifiable intregral-type CQ , . . . 93
CQFon05 CQ from [Fon05], . . . 105
CQHI CQ for problems with higher index, . . . 106
CQEHI Easier verifiable CQ for problems with higher index, . . . 106
CQ to guarantee Normality
CQ3FV94 CQ from [FV94], . . . 25 CQRV99 CQ from [RV99], . . . .26 CQCF05 CQ from [CF05], . . . .31 CQBF07 CQ from [BF07], . . . 32 CQ1n CQ involving ¯u, . . . .36 CQ2n CQ without ¯u, . . . 37 CQnVL CQ via linearization, . . . 44Calculus of Variations Problems
CQ4FV94 CQ from [FV94], . . . 373.1 Constraint qualification CQ1d (adapted from [Fon99]). . . 33
3.2 Constraint qualification CQ2d (adapted from [Fon99]). . . 33
6.1 Graphic representation of l exceeding any δ we might choose. . . 65
6.2 Graphic representation of R0tl(s)ds and −δt for a particular δ. . . 66
9.1 A higher index constrained system (from [Fon05]). . . 103
10.1 The connection between the results of Chapters 6 to 8. . . 115
Introduction
1.1
Scope and Motivation
In this thesis we deal with both the “Calculus of Variations Theory” and the “Opti-mal Control Theory”. Our study focus on a set of conditions (necessary conditions of optimality — NCO) that allow identify a small set of candidates to minimizers among the overall set of admissible solutions.
In the 1950’s, these conditions were proved for problems with high regularity on the data, but the continuous developments in this area allowed establishing condi-tions for problems with: “nonsmooth” data (data that can be non differentiable), more general end-point constraints, state constraints, and other refinements.
Almost all optimization problems arising in practice really have constraints and these constraints are limitations on our decisions. For example, operations may be limited to so many hours in day, a plane to fly in security must have constraints on altitude or velocity, chemical reactors have to be limited by maximum temperature or pressure, a vehicle or robot has to avoid obstacles, amongst many others. However, for optimal control problems with state constraints the standard NCO could not, in same cases, provide useful information to select candidates to minimizers. This happens when the set of candidates to minimizers that satisfy certain NCO coincides with the set of all admissible solutions or when the scalar multiplier associated
with objective function is equal to zero. It is possible, nevertheless, to avoid such phenomenon by strengthening the NCO.
As in [Fon99], we emphasise the importance attached to nondegenerate condi-tions by reference to their history in Mathematical Programming ([Aba67, Man69]). The Kuhn-Tucker conditions are best known optimality conditions for Mathemati-cal Programming problems with inequality constraints. However, these conditions are just a strengthened version of previous Fritz John conditions, imposing the mul-tiplier associated with the objective function to be positive, or simply equal to 1. Nowadays, the Kuhn-Tucker conditions are one of the most cited results in opti-mization. This illustrates the significance of nondegenerate versions of necessary conditions of optimality.
On other hand, the nondegenerate and normality results are important to estab-lish the regularity of optimal trajectories and controls, and also in estabestab-lishing links between NCO and Hamilton-Jacobi equations.
In this thesis we developed new strengthened forms (nondegenerate and normal forms) of necessary condition of optimality for optimal control problems with state constraints.
1.2
Overview
This thesis is organized as follows. In Chapter 2, we introduce the classical necessary conditions for calculus of variations and optimal control problems. We also introduce here some recent developments that will be of use later in this thesis and we finish this chapter with some concepts of regularity.
In Chapter 3, we review the main literature of strengthened necessary conditions for mathematical programming and optimal control problems.
Chapter 4 contains the normality result for calculus of variations problems that was developed in the author’s master thesis and a discussion of the relative merits of necessary conditions of optimality that were developed for optimal control problems, in the particular case of calculus of variations problems.
The main contributions of this thesis are given from Chapter 5 to Chapter 9. Chapter 5 involves a new normality result for optimal control problems via a lin-earization of control systems, while the results introduced on Chapter 6 to Chapter 9 are nondegenerate results.
In Chapter 6, we propose a nondegenerate maximum principle (MP) valid under a constraint qualification of integral-type.
The nondegenerate MP, provided in the Chapter 7, is valid under constraints qualifications that are easier to verify than some appearing in previous literature.
In Chapter 8, the main result guarantees the nondegeneracy for problems that satisfies an easier verifiable integral-type constraint qualification.
In the Chapter 9, we developed a new constraint qualification for optimal control problems with state constraints that have higher index (i.e. their first derivative with respect to time does not depend on the control).
We conclude this thesis by providing a summary of contributions and posing some related open questions to motivate further research.
Finally, we offer in the Appendix a brief review of relevant background material in functional analysis and nonsmooth analysis.
Background
Since necessary conditions of optimality (NCO) are the main tools of this thesis, we present in this chapter classical results on the subject for calculus of variations and optimal control problems in an informal setting. We also introduce some recent developments that will actually be of use in this thesis. We finish this chapter with some concepts of regularity.
2.1
NCO for Calculus of Variations Problems
The basic calculus of variations problem (CVP) is to find an absolutely continuous function ¯x that solves the following problem:
(CV P1) Minimize J [x] = Z t1 t0 L(t, x(t), ˙x(t))dt subject to x(t0) = x0 x(t1) = x1.
The interval [t0, t1], the Lagrangian function L : [t0, t1] × Rn× Rn → R, the initial
state x0 and the final state x1 are given as part of the problem statement.
We say that x is an admissible trajectory if x is an absolutely continuous function on the interval [t0, t1], satisfying the constraints of the problem, x(t0) = x0 and
x(t1) = x1 and such that L(t, x(t), ˙x(t)) is a Lebesgue integrable function in this
interval. The minimizer for the problem is an admissible trajectory ¯x in the interval [t0, t1], that satisfies
J [¯x] ≤ J [x], for any admissible trajectory x in [t0, t1].
The Calculus Variations theory is an important tool in laws of physics that iden-tified states of nature with minimizing curves and surfaces lengthened, as Fermat’s principle, Dirichlets’s principle, principle of least actions, among others, (see for example [Vin00] and [Loe93]).
The best known NCO for CVP are the Euler-Lagrange and the Weierstrass Condition (see for example [Cla89], [Vin00]). They assert the existence a function p ∈ W1,1([t0, t1] : Rn) such that Euler-Lagrange Condition: ( ˙p(t), p(t)) = Lx,u(t, ¯x(t), . ¯ x (t)), Weierstrass Condition: p(t)·x (t) − L(t, ¯¯. x(t),x (t)) = max¯. u∈Rn[p(t) · u − L(t, ¯x(t), u)].
2.2
NCO for Optimal Control Problems
2.2.1
The Problems
From a modern perspective, optimal control is a generalization of the calculus of variations.
As the name indicates, optimal control problems involve a control variable. In these problems the minimum cost depends both on the state and control variable. The control may be restricted to take values on a general set. The freedom to specify the set of possible controls combined with possibility of dealing with general cost functions covers a wide range of control engineering problems.
The mathematical formulation of a OCP appears in three forms: Bolza, Lagrange and Mayer problems.
We start by introducing the Bolza problem, as following:
(OCPB) Minimize g(x(t0), x(t1)) + Z t1 t0 L(t, x(t), u(t))dt subject to ˙x(t) = f (t, x(t), u(t)) a.e.t ∈ [t0, t1]
(x(t0), x(t1)) ∈ C
u(t) ∈ Ω(t).
The data for this problem comprise functions g : Rn× Rn → R, L : [t
0, t1] × Rn×
Rm → R, f : [t0, t1] × Rn× Rm → Rn, a closed set C ⊂ Rn× Rn and a multifunction
Ω : [t0, t1] Rm.
The function to minimize
g(x(t0), x(t1)) +
Z t1
t0
L(t, x(t), u(t))dt (2.1)
is known as cost function.
The variable x is called the state. The function describing state time evolution, x(t), t0 ≤ t ≤ t1 is called state trajectory.
The set of control functions for (OCPB), denoted U , is the set of measurable
functions u : [t0, t1] → Rm such that u(t) ∈ Ω(t) a.e. t ∈ [t0, t1].
The domain of the above optimization problem is the set of admissible processes, namely pairs (x, u) comprising a control function u and a corresponding state tra-jectory x which satisfy the constraints of (OCPB).
If the cost function (2.1) is simply •
Z t1
t0
L(t, x(t), u(t))dt, then the problem is known as Lagrange problem; • g(x(t0), x(t1)), then the problem is known as Mayer problem.
The Bolza problem can be transformed in these two special problems, Lagrange and Mayer problem, by adding a new state variable, (see for example: [PF62], [Tor02]).
Example of optimal control problems in three forms can be found in [Ber95]. According to [Vin00], the importance of Mayer formulation is that it embraces a wide range of significant optimization problems which are beyond the reach of traditional variational techniques and it is very well suited to the derivation of general necessary conditions of optimality. In next chapters, we consider OCP in Mayer form.
Additional constraints can be added to the problem. For example: • equality state constraint:
k(t, x(t)) = 0 for t ∈ [t0, t1], for a given function k : [t0, t1] × Rn → R;
• inequality state constraint(a):
h(t, x(t)) ≤ 0 for t ∈ [t0, t1], for a given function h : [t0, t1] × Rn → R;
• implicit state constraint(b):
x(t) ∈ X(t) for t ∈ [t0, t1], in which X : [t0, t1] Rn is given multifunction;
• mixed state constraint :
g(t, x(t), u(t)) ≤ 0 for a.e. t ∈ [t0, t1], in which g : [t0, t1] × Rn× Rm → Rk
is given.
State constraints(a),(b)are object of study in this thesis. Problems with the mixed
state constraint are considered in [Aru00], [dP03], and [MdRdPZ01].
Here, we consider fix-time problems. However, free-time problem could be con-sidered, where the problem is defined on an interval [t0, t0 + T ] and it is desired
minimize time T (see for example [PF62],[Ber95]). These problems are known as minimal time problems.
2.2.2
Maximum Principle
The NCO for OCP appear in the form of Maximum Principle (MP). It is usually accepted that the MP was introduced by Pontryagin and his collaborates in the
paper [PBGM62].
The original formulation of the MP applied to problems with very basic restric-tions and with smoothness hypotheses.
Assuming that, the (pseudo-) Hamiltonian function 1 is defined as follows:
H(t, x, p, u) = p · f (t, x, u) − λL(t, x, u).
The MP under smoothness hypotheses, states that if (¯x, ¯u) is a minimizer of (OCPB), then there exists an absolutely continuous function p and λ ≥ 0, not both
zero, such that the following conditions are satisfied: The Adjoint Condition:
− ˙p(t) = Hx(t, ¯x(t), p(t), ¯u(t)) a.e.;
The Maximum Principle: ¯u(t) maximizes over Ω(t) the function
u → H(t, ¯x(t), p(t), u) a.e.;
The Transversality Condition:
(p(t0), −p(t1)) − λgx(¯x(t0), ¯x(t1)) is normal to C at (x(t0), x(t1)).
A brief historical survey of NCO for optimal control and calculus of variations problems can be found in [Sar00].
2.3
Nonsmooth NCO
Optimization problems in which the cost function to minimize is not differentiable appear frequently. Two simple examples of nondifferentiable functions are:
1What Hamilton really defined was the “maximized” hamiltonian H(t, x, p) = p · v(x, p) −
• functions that state lengths and distances;
• function defined as the max or min of a collection of differentiable functions. Others examples of problems with nonsmooth data can be found, for example, in [Cla83].
Nowadays, there exists a great interest in developing necessary conditions for problems with nonsmooth data.
2.3.1
Nonsmooth NCO for Calculus of Variations Problems
An extension of the Euler-Lagrange (see for example [Cla89]), allowing nonsmooth data is
( ˙p(t), p(t)) ∈ ˜∂L(t, ¯x(t),x (t)) a.e..¯.
Here, ˜∂L denotes the Clarke’s subdifferential with respect to (x, u).
If function f : Rn → R is Lipschitz continuous on a neighborhood of a point
x ∈ Rn, the Clarke’s subdifferential is given by
˜
∂f (x) = co {η ∈ Rn : ∃xi → x, xi ∈ Ω, f/ x(xi) exist and fx(xi) → η},
where Ω ⊂ Rn having Lebesgue measure zero.
We have defined this subdifferential only for Lipschitz continuous function, how-ever Clarke provided an extension to lower semicontinuous functions, see [Cla89].
If L is continuously differentiable, then ˜∂L(t, ¯x(t),x (t)) reduces to the singleton¯. set {Lx,u(t, ¯x(t),
.
¯ x (t))}.
2.3.2
Nonsmooth NCO for Optimal Control Problems
In this section, we introduce NCO for OCP in Mayer form with endpoint state constraints. Without loss of generality, we consider the interval [0, 1] as the “time” domain of our problem. The problem of interest is:
(OCPM 1) Minimize g(x(0), x(1))
subject to ˙x(t) = f (t, x(t), u(t)) a.e.t ∈ [0, 1] x(0) ∈ C0
x(1) ∈ C1
u(t) ∈ Ω(t) a.e.t ∈ [0, 1]. The data for this problem comprise functions g : Rn× Rn
→ R, f : [0, 1] × Rn×
Rm → Rn, the sets C0 and C1 and a multifunction Ω : [0, 1] Rm.
Remark 2.3.1 (On Differential Inclusions) The control system ˙x(t) = f (t, x(t), u(t)) u(t) ∈ Ω(t) (2.2) can be interpreted as ˙x(t) ∈ F (t, x(t)) a.e., (2.3)
in which, for each (t, x), F (t, x) is a given subset of Rn.
If F (t, x(t)) = f (t, x(t), Ω(t)), the set of solutions to (2.2) coincides with set of solutions to differential inclusion (2.3), under the following mild hypotheses on the data for (OCPM 1), (see [Vin00], pag.73):
(i) f (·, x, ·) is L × Bm measurable and f (t, ·, u) is continuous;
(ii) Gr Ω is L × Bm measurable.
Remark 2.3.2 (On minimizer) When we seek a solution of an optimal control problem, we must specify if we are looking for a local or a global minimizer. The meaning of local needs to be clarified. Different choices of topology on the set of admissible processes give rise to different notions of local minimizer.
Throughout this thesis, we say that an admissible process (¯x, ¯u) is a local mini-mizer if there exists δ > 0 such that
for all admissible processes (x, u) satisfying
kx(t) − ¯x(t)kL∞ ≤ δ.
For (OCPM 1), the Hamiltonian function is
H(t, x, p, u) = p · f (t, x, u).
We provide here a version of MP under minimum hypotheses in which it makes sense to talk about a OCP, as Clarke mentions in the paper [Cla76a]. They are denoted here and throughout as the Basic Hypotheses.
Theorem 2.3.3 Let (¯x, ¯u) be a local minimizer for (OCPM 1). Assume that, for
some δ0 > 0, the following Basic Hypotheses are satisfied.
H1b The function (t, u) → f (t, x, u) is L × Bm measurable for each x. (L × Bm
denotes the product σ-algebra generated by the Lebesgue subsets L of [0, 1] and the Borel subsets of Rm.)
H2b There exists a L × Bm measurable function k(t, u) such that t 7→ k(t, ¯u(t)) is
integrable and
kf (t, x, u) − f (t, x0, u)k ≤ k(t, u)kx − x0k
for x, x0 ∈ ¯x(t) + δ0B, u ∈ Ω(t) a.e.t ∈ [0, 1].
H3b The function g is Lipschitz continuous on ¯x(1) + δ0B.
H4b The graph of Ω is L × Bm measurable.
H5b The sets C0 and C1 are closed.
Then there exist p ∈ W1,1([0, 1] : Rn) and λ ≥ 0 such that
− ˙p(t) ∈ co ∂L
xH(t, ¯x(t), p(t), ¯u(t)) a.e.t ∈ [0, 1],
(p(0), −q(1)) ∈ NCL0(¯x(0)) × NCL1(¯x(1)) + λ∂Lg(¯x(0), ¯x(1)), (2.4) and for almost every t ∈ [0, 1], ¯u(t) maximizes over Ω(t)
u → H(t, ¯x(t), p(t), u).
Remark 2.3.4 Here, co C is the convex hull of a set C ⊂ Rn. The set NL C(x) is
the limiting normal cone to the closed set C ⊂ Rn at x ∈ C defined as
NL
C(x) = {η ∈ Rn : ∃ sequences {Mi} ∈ R+, xi → x, ηi → η such that
xi ∈ C and ηi· (y − xi) ≤ Miky − xik2 for all y ∈ Rn, i = 1, 2, ...}.
Let f : Rn → R ∪ {+∞} be a lower semicontinuous function and x ∈ dom f.
Then the set ∂Lf (x) defined as
∂Lf (x) = {η ∈ Rn : (η, −1) ∈ Nepi fL (x, f (x))},
where epi f = {(x, α) ∈ Rn+1 : α ≥ f (x)}, is the limiting subdifferential of f at x.
2.3.3
Nonsmooth NCO for Optimal Control Problems with
State Constraints
In this section, we introduce the MP for an OCP with inequality state constraints, as the following: (OCPM 2) Minimize g(x(0), x(1))
subject to ˙x(t) = f (t, x(t), u(t)) a.e.t ∈ [0, 1] x(0) ∈ C0
x(1) ∈ C1
u(t) ∈ Ω(t) a.e.t ∈ [0, 1] h(t, x(t)) ≤ 0 for all t ∈ [0, 1].
In [HSV95], we can find a survey of MP for problems with state constraints. There references to the direct adjoint approach, indirect adjoint approach, and methods that use transformations converting problems with state constraints into problems without state constraints are made. However, problems with nonsmooth data are not addressed.
The NCO for nonsmooth and state constraints OCP were introduced in [VP82]. This result generalized MP introduce by [Cla76a], by allowing state constraints in the form
h(t, x(t)) ≤ 0, for all t ∈ [0, 1]. (2.5) They show that Clarke’s methodology can be adapted to permit such constraints. The underlying idea is to replace constraints (2.5) by a penalty term added to the cost g(x(0), x(1)) + k Z 1 0 max{0, h(t, x(t))}dt, for some k > 0.
Nonsmooth MP for state constrained problems are also proved in [Cla83] and [VZ98].
necessary conditions.
Assume that, in addition to H1b-H5b, the following hypothesis are imposed on
(OCPM 2):
H6b The function h is upper semicontinuous in t and there exists a scalar Kh > 0
such that the function x → h(t, x) is Lipschitz of rank Kh for all t ∈ [0, 1].
Then the MP is stated in the following form:
Theorem 2.3.5 ([Vin00]) If (¯x, ¯u) is an local minimizer, then there exist p ∈ W1,1([0, 1] : Rn), measurable function γ, a nonnegative Radon measure µ ∈ C∗
([0, 1], R) and a scalar λ ≥ 0 such that
µ{[0, 1]} + kpkL∞ + λ > 0, − ˙p(t) ∈ co ∂L xH(t, ¯x(t), q(t), ¯u(t)) a.e.t ∈ [0, 1], (p(0), −q(1)) ∈ NCL0(¯x(0)) × NCL1(¯x(1)) + λ∂Lg(¯x(0), ¯x(1)), γ(t) ∈ ∂x>h(t, ¯x(t)) µ − a.e., supp {µ} ⊂ {t ∈ [0, 1] : h(t, ¯x(t)) = 0}, (2.6) and, for almost every t ∈ [0, 1], ¯u(t) maximizes over Ω(t),
u → H(t, ¯x(t), q(t), u). where q(t) = p(t) + Z [0,t) γ(s)µ(ds) t ∈ [0, 1) p(t) + Z [0,1] γ(s)µ(ds) t = 1. Here, ∂>
xh(t, x), denotes the hybrid partial subdifferential of h in the x-variable
defined as ∂>
xh(t, x) = co{ξ : there exist (ti, xi) → (t, x) s.t. h(ti, xi) > 0,
The condition (2.6) is denoted by “Complementary Slackness Condition”; it states that µ is equal to zero if the state constraint is inactive at x on t (i.e. h(t, x(t)) < 0).
Note that if the state constraint is inactive at x, then the statement of the theorem simplifies due to the fact that all mention to µ (and the corresponding integrals) may be removed.
It is worth mentioning that introduction of chapter 9 in the book of [Vin00], we can find the key ideas behind the derivation of NCO for problem with state constraints.
2.4
Existence and Regularity
Application of NCO to identify a set of candidates to the optimal solution only make sense if the optimal solution exists. Therefore, there is great interest in studding the existence of optimal solutions.
It was Tonelli (1915) who introduced the first theorem of the existence of solution for CVP. Even today, the Tonelli’s theorem remains the central existence theorem for CVP, although the hypotheses of the theorem can be relaxed, see for example [Vin00]. For OCP, results that guarantee the existence of solution can be found in [Cla83], for example.
The hypotheses under which existence of an optimal solution may not coincide with those under which NCO are valid.
A simple example of that occurs in calculus of variations: the Tonelli’s theorem guarantee the existence of minimizers in the class of absolutely continuous functions, whereas the Euler-Lagrange condition is applied for arcs with essentially bounded derivatives.
Regularity analysis helps us to identify classes of problems, for which all mini-mizers satisfy known NCO. This analysis seeks information about regularity of min-iminizers, for example when the minimizers arcs are Lipschitz continuous (we call Lipschitz regularity), minimizers arcs with higher-order derivatives or the optimal
control that are Lipschitz continuous.
In recent years many authors got interested on the study of the Lipschitz regu-larity of the optimal trajectory, because of its important implications. In Control Engineering, this regularity condition allows to compute the true optimal trajectory by numerical methods. On other hand this condition ensures the non-occurrence of the Lavrentiev phenomenon - the infimum cost over the space of absolutely con-tinuous functions is strictly less than the infimum cost over the space of Lipschitz continuous functions. A simple example in which this phenomenon occurs was given by Mani´a, (see for example [Cla89]).
Many authors contributed to the investigation of Lipschitzianity of optimal tra-jectories for CVP, see for example [CV85] and [Vin00]. Less is known for OCP. In this respect we refer the reads to [ST00] and [GV03] for OCP, (where the con-trolled differential equation is linear in the control variable), the result of [DK95] and [CLV97] for linear quadratic problems with state constraints. However, Lips-chitz regularity of the optimal trajectory for nonlinear OCP with state constraints is still an open question.
The Degeneracy Phenomenon of
Necessary Conditions of
Optimality
In this chapter, we discuss the degeneracy phenomenon in optimization problems with inequality constraints. We start by describing this phenomenon in the context of mathematical programming problems, recalling the Fritz-John and Kuhn-Tucker conditions. Later, we address the degeneracy phenomenon in the context of optimal control problems. We review and discuss nondegenerate necessary conditions of optimality for optimal control problems with state constraints. An overview of the main literature in this area is made, including a comparison with some recent results from the authors.
3.1
Degeneracy in Mathematical Programming
The general mathematical programming problem (MPP) consists in minimizing a given function f (x) subject to three types of constraints: inequality constraints, equality constraints and implicit state constraints. Here, we consider the MPP with
inequality constraints: (M P P1) Minimize f (x) subject to gi(x) ≤ 0, i = 1, 2, ...., n.
Throughout this section, we assume that the functions f and gi for each i = 1, 2, ...., n are continuously differentiable.
If ¯x is a solution to the problem (M P P1), then the NCO in the form of Fritz-John
conditions [Joh48] in [BSS93] guarantee the existence of nonnegative multipliers λ and µi, with i = 0, 1, 2, ..., n such that
(λ, µ1, ..., µn) 6= 0 (3.1) λfx(¯x) + n X i=1 µigxi(¯x) = 0 (3.2) µigi(¯x) = 0, for i = 1, ..., n. (3.3)
If the second condition is satisfied with λ = 0, the cost function is not involved in the choice of candidates to minimizers. So, the NCO does not give any information about the candidate to minimizers and the NCO are merely a relation between the constraints. When this happens, we say that the NCO are degenerated.
A way of forcing the cost function to be involved in the NCO is to assume that λ = 1 on the conditions (3.1)-(3.3), known as normal form of the NCO. However, we have to guarantee that the NCO are still satisfied at local minimum. If it is not the case, the NCO are not valid. So additional hypotheses, known as Constraint Qualification (CQ), are considered to identify the problems under which the normal form is ensured.
Some of the best known examples of a CQ are:
Linear Independence CQ: for every local minimizer ¯x, the gradients of the active constraints are linearly independent;
v ∈ Rn such that
gix(¯x) · v < 0 if gi(¯x) = 0, i = 1, 2, ..., n.
Another type of CQ, called “calmness”, was introduced in [Cla73], see [Cla83]. Assume that ¯x is minimizer to (M P P1) and P (p) is the problem of minimizing
f (x) over points x ∈ Rn which satisfy the constraints g(x) + p ≤ 0. The (M P P1) is
calm at ¯x provided that there exist positive ε and M such that, for all p ∈ εB, for all x0 ∈ ¯x + εB which are feasible for P (p), one has
f (x0) − f (¯x) + M kpk ≥ 0.
The calmness of MPP at ¯x allows to write the NCO (3.1)-(3.3) with λ = 1. The Kuhn-Tucker conditions [KT51] are precisely a normal version of the Fritz John conditions valid under a suitable CQ. They state that the conditions (3.1)-(3.3) can be written with λ = 1 for all problems complying with the CQ.
The work of Kunh and Tucker, probably one the most cited results in optimiza-tion, is in fact a strengthened and nondegenerate form, of the Fritz John conditions. This fact justifies the importance of studying nondegenerate versions of NCO for constrained optimization problems. This problem is well-studied in the context of mathematical programming for along time. However, the degeneracy phenomenon in the OCP context has witness many important advances in the very recent years.
3.2
Degeneracy in Optimal Control Problems
In this section, we discuss strengthened forms of MP for OCP, like (OCPM 2), which
guarantee nondegeneracy and/or normality.
The term “degeneracy” has been used in optimal control literature to describe a particular type of degeneracy occurring due to the presence of pathwise state constraints which are active at the initial time. Assuming that the pathwise state
constraint is active in the initial instant of time, i.e.
h(0, x0) = 0, (3.4)
the set of multipliers (degenerate multipliers)1
λ = 0, µ = δt=0, p = −hx(0, x0) (3.5)
satisfies the NCO for all admissible process (x, u). This can be easily seen by noting that the quantity p(t) +
Z
[0,t)
hx(s, ¯x(s))µ(ds) vanishes almost everywhere and all
conditions of the MP, (Theorem 2.3.5), are satisfied independently of the value of ¯x or ¯u. In this case, the NCO are said to be degenerate.
In this thesis we will be concentrated in this kind of degeneracy. However other type of degeneracy can occur, namely “the q-degeneracy” (see for example [Fon99]). The case (3.4) is encountered in certain applications of interest, namely Model Predictive Control. A further discussion of this point can be seen in [FV94, Fon99]. In order to avoid the degeneracy, the MP can be strengthened with additional conditions, typically a strengthened form of the nontriviality condition.
The term normality is used when the MP for OCP can be written with the multiplier associated with the objective function λ not zero.
Definition 3.2.1 (Normality) An optimal control problem is said normal if the con-ditions of Theorem 2.3.5 are satisfied with λ = 1.
.
The normality and regularity 2 are closely connected.
In [Fer06], it is proved that the conditions imposed to get the Lipschitz continuity of the optimal control may also contribute to guarantee the normality of MP.
Results where Lipschitz regularity is ensured as a consequence of normal NCO, can be found in [FM06].
1Here δ
{0} denotes the unit measure concentrated at {0}. 2The term regularity as the same meaning as in section 2.4.
Normal necessary conditions have been developed for problems with nonsmooth as well as smooth data, problems in which the dynamic constraints involves a dif-ferential inclusion, or a difdif-ferential equation, and in which the state constraint is formulated as a set inclusion as well as a functional inequality.
In next section, we make an overview of the main literature in these area.
3.2.1
Avoiding the Degeneracy Phenomenon
Calmness
As in mathematical programming, the new type of CQ introduce by Clarke “calm-ness” allow to strength the MP with λ = 1.
For the problem
(OCPM 3) Minimize g(x(0), x(1)) subject to ˙x(t) ∈ F (t, x(t)) a.e., calmness is defined as follows:
Definition 3.2.2 Let φi : Rn→ [−∞, ∞], i = 0, 1, be defined as
φ0(s) = inf{g(x(0) + s, x(1)) : ˙x ∈ F (t, x(t)) a.e. },
φ1(s) = inf{g(x(0), x(1) + s) : ˙x ∈ F (t, x(t)) a.e. }.
Then, problem (OCPM 3) is said to be calm if, for i = 0 or 1,
φi = lim inf
s→0 φ
i(s) − φi(0) /|s| > −∞.
As shown in [Cla76b], calmness allows to write the MP with λ = 1, when F (t, x) is measurable in t and Lipschitz in x near ¯x and g : Rn× Rn → (−∞, ∞] is lower semicontinuous. However, pathwise state constraints are not considered.
In the remaining of these sections, we consider OCP with pathwise state con-straints: inequality constraints or implicit constraints.
Nondegenerate Result from [AA97]
In [AA97], a new MP is developed to avoid the degeneracy, for Lipschitz continuous trajectories where the problem is:
(OCPM 3) Minimize g(x(0), x(1)) subject to ˙x(t) ∈ F (t, x(t)) a.e.t ∈ [0, 1] (x(0), x(1)) ∈ C0× C1 x(t) ∈ X ∀t ∈ [0, 1],
The MP contains additional information about the behavior of the Hamiltonian at the endtimes: ˜ H t, ¯x(t), p(t) + Z [0,t) dµ = ˜H t, ¯x(t), p(t) + Z [0,t)∪{t} dµ (3.6)
for all t ∈ [0, 1] where:
• ˜H(t, x, q) := maxf ∈F (t,x)q · f is the true (maximized) Hamiltonian;
• supp µ ⊂ {t ∈ [0, 1] : x(t) ∈ bdy (X)}; µ(t) ∈ NX(¯x(t)) ∀t ∈ [0, 1].
(NX is Clarke normal cone)
The condition (3.6), combined with the following constraint:
CQAA97
˜
H(0, x0, −g) > 0,
∀g ∈ NX(¯x(0)) ∩ N(CL0∩X)(¯x(0)). eliminates the degenerate multipliers.
Loosely speaking, CQAA97 requires the existence of a control function pulling
the state away from the state constraint boundary at the initial time.
For the results in [AA97] to be valid, it is required that the multivalued mapping F is locally Lipschitz with nonempty convex compact values.
Nondegenerate Result from [FV94]
Another result to avoid degeneracy is developed in [FV94]. It also required f (t, x, Ω(t)) to be convex but data are merely required to be measurable in time. For a problem like (OCPM 2) (see section 2.3.3) whith initial state fixed and free final state, the
nondegeneracy NCO are strengthened with the nontriaviality condition Z
(0,1]
µ(ds) + λ > 0,
if one of the following CQ are satisfied: CQ1FV94: there exists a control ˜u such that
hx(t, x0) · [f (t, x0, ˜u) − f (t, x0, ¯u(t)] < 0,
for t near 0 (that means, there exits control function pulling the state away from the boundary of the state constraint set faster than the optimal control); CQ2FV94: : there exists ¯t ∈ (0, 1] such that h(t, ¯x(t)) < 0, ∀t ∈ (0, ¯t], (that means,
the minimizing trajectory itself leaves the boundary immediately).
Conditions to ensure normality are described in terms of the dynamic equa-tions, linearized with respect to the state variables. The constraints qualifications CQ1FV94 and CQ2FV94 are strengthened with the following condition:
CQ3FV94:
hx(t, ¯x(t)) · yu(t) < 0 ∀t ∈ (0, 1] ∩ {t ∈ [0, 1] : h(t, ¯x(t)) = 0},
where yu is the unique absolutely continuous function satisfying:
˙
yu(t) = fx(t, ¯x(t), ¯u(t)) · yu(t) + f (t, ¯x(t), u(t)) − f (t, ¯x(t), ¯u(t)) a.e. t ∈ [0, 1]
yu(0) = 0,
(3.7) given a control u.
Nondegenerate Result from [FFV99]
The result in [FFV99] generalizes the nondegenerate result in [FV94] with CQ1FV94
by allowing the final state to belong to a given set C1, the data to be nonsmooth
and by not requiring the velocity set f (t, x, Ω(t)) to be convex. In this paper new methods are introduced for proving nondegenerate NCO. The key idea of the proof is to replace the original control problem by one in which the state constraints is eliminated on [0, α], for arbitrary small α.
The multipliers of the MP for this new problem are nondegenerate. Passing to the limit α ↓ 0 we concluded that the limiting multipliers are nondegenerate and the nontriviality condition can be replaced by
µ{(0, 1]} + kqkL∞+ λ > 0.
Normality Result from [Fon00]
Based on nondegeneracy results in [FFV99], [Fon00] ensures the normality of the MP for free final state problem, if there is a control that can pull the trajectory away from the boundary (faster than the optimal control) for every instant that inequality constraints is active.
In the works mentioned above ([FV94], [FFV99], and [Fon00]), the conditions involve the minimizing ¯u which we do not know in advance, and consequently the conditions are, in general not easily verifiable, except in special cases, such as CVP. (See next chapter)
Normality Result from [RV99]
Nondegenerate NCO for OCP valid under a CQ that no longer involve the minimiz-ing ¯u, appear in [RV99]. The MP can be written with λ = 1, if
CQRV99: there exists a continuous feedback u = η(t, ξ) such that
for some positive δ, whenever (t, ξ) is close to the graph of ¯x(·) and ξ is near the state constraint boundary.
The problem considered is (OCPM 2), but the functions defining the dynamics is
now Lipschitz continuous with respect to time, the final state is free, and the initial state belongs to a given set C0.
The proof of existence of normal multipliers is based on a main theorem, called neighbouring feasible trajectories theorem. It asserts that for a prespecified process which may violate the state constraint there exists another process that it is suitably close to the first one and satisfies the state constraint.
Nondegenerate Result from [RV00]
Building upon their neighbouring feasible trajectories theorem, [RV00] derived non-degenerate NCO which apply to differential inclusion problems (OCPM 3) where the
state constraints set X takes the form:
X =
m
[
j=1
{x : hj(x) ≤ 0}
for some functions hj : Rn→ R, j = 1, ..., m of class C1,1.
Assuming that the velocity set F (t, x) is nonconvex and measurable in time, the NCO are strengthened with the nontriviality condition
λ + Z (0,1] X j µj(ds) + |p(0) +X j hjx(¯x(0))µj({0})| 6= 0,
when subject to follow constraint qualification:
CQRV00: For each t ∈ [0, ] and ξ ∈ ¯x(0) + δ0B
minv∈F (t,ξ)hjx(ξ) · v < −δ
Nondegenerate Result from [CF05]
In the paper [CF05], we can find a strengthened MP for (OCPM 3) with dynamics
given by a nonconvex differential inclusion and fixed initial state.
To derive these results, it was necessary impose the following hypotheses: Hypothesis 3.1.
i) F (·, ·) : [0, 1] × Rn Rn is a multifunction with nonempty closed values. ii) ∀x ∈ Rn, F (·, x) is measurable.
iii) There exists c > 0 such that ∀(t, x) ∈ [0, 1] × Rn, F (t, x) ⊂ c(1 + kxk)B.
iv) There exists l(·) ∈ L1 such that F (t, ·) is l(t)- Lipschitz continuous. v) g : Rn→ R is locally Lipschitz.
Hypothesis 3.2. (Used to establish the existence of a “linearization” of F along (¯x,x) by closed convex processes, which are Lipschitz with respect to the state.) There¯. exists of a family of closed convex process A(t, ·) : Rn
Rn, t ∈ [0, 1], that satisfies i) A(·, v) is measurable ∀v ∈ Rn.
ii) A(t, v) ⊆ ¯dxcoF (t, ¯x(t), .
¯
x (t))v ∀v ∈ Rn for a.e. t ∈ [0, 1].
iii) For some m ≥ 0, A(t, ·) is m-Lipschitz on Rn for a.e. t ∈ [0, 1]. ( ¯dxF (·) is the adjacent derivative of coF (t, ·) at (¯x(t),
.
¯
x (t)), see appendix.)
Hypothesis 3.3. (Used to the existence of a convex “linearizations” of con-straints along optimal trajectories is also considered.) X and C1 are closed subsets
of Rn, Int (C
C1(¯x(1))) 6= ∅ and there exists a lower semicontinuous multifunction G : [0, 1] Rn such that for all t ∈ [0, 1], G(t) is a closed convex cone with
nonempty interior and for every v ∈ Int(G(t)) we can find ε > 0 such that for all s ∈ [t − ε, t + ε] ∩ [0, 1], ¯x(s) + [0, ε](v + εB) ⊂ X.
Theorem 3.2.3 Let ¯x(·) be an optimal solution to (OCPM 3) with initial state fixed
assume that Hypotheses 3.1-3.3 hold true. Further assume that an upper semi-continuous concave positively homogeneous function ψ : Rn → R ∩ {−∞}
satis-fies Int (G(0)) ⊂ dom(ψ) and ψ ≤ Dx+V (0, ¯x(0)). Then there exits λ ∈ {0, 1}, ψ ∈ N V B([0, 1]) and an absolutely continuous function p(·) : [0, 1] → Rn such that
λ + kψkT V 6= 0 and p satisfies the
˙ p(t) ∈ A∗(t, −p(t) − ψ(t)) a.e. in [0, 1] p(1) ∈ −λ ˜∂g(¯x(1)) − ψ(1) − NC1(¯x(1)), (p(t) + ψ(t))·x (t) =¯. max v∈F (t,¯x(t)) (p(t) + ψ(t)) · va.e. in [0, 1] −p(0) ∈ λ∂+ψ(0). Furthermore, ψ(0+) ∈ G(0)−, ψ(t) − ψ(t−) ∈ G(t)−, ψ(t) = Z [0,t] ν(s)dµ(s) ∀t ∈ (0, 1]
for a positive (scalar) Randon measure µ on [0, 1] and a µ-measurable function ν(·) : [0, 1] → Rn satisfying
ν(s) ∈ G(s)−∩ B µ − a.e..
If CC1(¯x(1)) ∩ Int (G(1)) 6= ∅, then the following non degeneracy condition holds true
λ + sup
t∈(0,1)
kp(t) + ψ(t)k 6= 0 (3.8)
and if ¯x(1) ∈ Int (C1), then
λ + var(ψ, (0, 1]) 6= 0, (3.9)
Moreover λ = 1 if there exists a solution to the constrained differential inclusion ˙ w(t) ∈ A(t, w) + Tco(F (t,¯x(t)))( . ¯ x (t)), (3.10) satisfying w(t) ∈ Int(G(t)) ∀t ∈ [0, 1], w(1) ∈ Int(CC1(¯x(1))). (3.11) Here:
• N V B([0, 1]) is the space of functions f of bounded variation on [0, 1], which are continuous from the right on (0, 1) and such that f (0) = 0;
• The norm of f ∈ N V B([0, 1]) is the total variation of f on [0, 1] denoted by kf kT V;
• G− is the negative polar cone of G;
• NX(x) is the Clarke normal cone to the set X at x ∈ X;
• Tco(F (t,¯x(t)))( .
¯
x (t)) denotes the tangent cone of convex analysis to co(F (t, ¯x(t))) at x (t);¯.
• V (·, ·) is the value function
V (t0, y0) = inf g(x(1)) : x(·) is the solution to ˙x(t) ∈ F (t, x(t)) (x(0), x(1)) ∈ C0× C1 x(t) ∈ X x(t0) = y0 on [t0, 1], • D+V (0, x
0)(·) denotes the upper derivative of V (0, ·) at x0;
• ∂+ψ(0) denotes the superdifferential of ψ at 0;
(Definitions can be found in Appendix.)
To prove Theorem 3.2.3, duality of convex analysis is applied. In this way they extend the known relations between the maximum principle and dynamic program-ming from the unconstrained problems to constrained cases, where the calmness of value function is used to investigate nondegeneracy of MP.
To allow to write the Theorem 3.2.3 with λ = 1, it was necessary assume the following additional hypothesis:
Hypothesis 3.4. Assume that for some η > 0 the signed distance
h(x) =
−dist(x, bdy (X)) ∀x ∈ X dist(x, bdy (X)) otherwise
is of class Cloc1,1 on bdy X + ηB and the following CQ :
CQCF05: there exists δ > 0 such that for almost all t ∈ [0, 1] with ¯x(t) ∈ bdy (X) +
ηB we have
min
v∈F (t,¯x(t))hx(¯x(t)) · v ≤ −δ.
Theorem 3.2.4 Let ¯x(·) be an optimal control solution to (OCPM 3) with initial
state is fixed assume that Hypotheses 3.1, 3.2 and 3.4 hold true and ¯x(1) ∈ Int (C1). Then all conclusions of Theorem 3.2.3 are valid with λ = 1 and G(t) =
TX(¯x(t)) for every t ∈ [0, 1].
The prove is based on ensured the existence of a function like (3.10) satisfying (3.11).
This result is similar to [RV99], however in this one the inward pointing condition is weaker condition, it has to be satisfy just along the optimal trajectory.
Normality Result from [BF07]
For a Bolza problem like (OCPB) with Lipschitz continuous trajectories, where
the initial state belongs to a given set, the final state is free and trajectories are constrained to a closed set x(t) ∈ X, the normality is ensured in [BF07].
This result is valid for problems that satisfy the following constraint qualification: CQBF07: ∃δR> 0 such that, ∃uy ∈ Ω(t) satisfying
supn∈C−
X(x)∩Sn−1n · f (t, y, uy) ≤ −δR,
where Sn−1 = {x ∈ R : kxk = 1} and CX−(x) is the negative polar of Clarke’s tangent cone to the set X at x. (see definition in Appendix)
The advantage of this result is that it allows nonsmooth and nonconvex state constraints.
Normality Result from [Mal03]
The normality for an optimal control problem with mixed control-state and pure state constraints is described in the work of Malanowski [Mal03]. The constraints qualifications involve the gradients of the constraints which are in some sense almost active and involve also the controllability of the linearized state equation. If the data are differentiable and the constraint qualification mention above is satisfied, then there exists an unique normal Lagrange multiplier.
Comments
In summary, the constraint qualifications found in the literature to avoid degeneracy in optimal control problems with state constraints can be divided into two types:
CQ1d: (from [FV94] and [FFV99]) ∃δ, > 0 and ∃˜u(t) ∈ Ω(t):
hx(x0) · [f (t, x0, ˜u(t)) − f (t, x0, ¯u(t)] < −δ a.e.t ∈ [0, ).
Loosely speaking, this is the requirement that there exist a control function pulling the state away from the boundary of the state constraint set faster than the optimal control on a neighborhood of the initial time. (see Figure 3.1)
Figure 3.1: Constraint qualification CQ1d (adapted from [Fon99]).
Figure 3.2: Constraint qualification CQ2d (adapted from [Fon99]).
CQ2d: (from [AA97] and [RV00]) ∃δ, > 0 and ∃˜u(t) ∈ Ω(t):
hx(x0) · f (t, x0, ˜u(t)) < −δ a.e.t ∈ [0, ).
Meaning, that there exits a control functions pulling the state away from the state constraint boundary on a neighborhood of the initial time. (see Figure 3.2)
Extending CQ1d and CQ2d, in such way that they are verifiable not only on
a neighborhood of the initial time, but also on neighborhood of each instant that the minimizer trajectory touches the boundary, allows to write the MP with λ = 1. Here, we denote by CQ1n and CQ2n (respectively), the constraint qualification
Clearly, a normal form of MP implies a nondegenerate form of MP. However most of these results require some regularity on data. See for example [FV94], [RV99], [Fon00], [CF05] and [BF07].
[RV99], [CF05] and [BF07] use constraints qualifications of the type CQ2n, while
[FV94] and [Fon00] use the constraint qualification of the type CQ1n. Comparing
theses results, we conclude that normal MP using constraint qualification of the type CQ1n, as in [Fon00], requires less regularity. However, CQ1n involves the
minimizing ¯u which we do not know in advance, and consequently the condition is, in general not easily verifiable.
Notes on Chapter
Normality in Calculus of
Variations Problems
In this chapter, we show how calculus of variations problems (CVP) can be seen as a particular case of an optimal control problems (OCP) and we study normality of necessary conditions of optimality (NCO) for CVP as a consequence of the normality of NCO for OCP.
4.1
Introduction
Throughout this chapter, we focus the following CVP subject to inequality states constraints: (CV P2) Minimize J [x] = Z 1 0 L(x(t), ˙x(t))dt subject to x(0) = x0 h(x(t)) ≤ 0 ∀t ∈ [0, 1].
Observe that the functional defining the state constraints does not depend ex-plicitly on t.
The special structure of CVP permits the derivation of constraint qualifications (CQ) that can be much easier to verify than in the optimal control case. Hence, the
interest in exploring dynamic optimization problems with this special structure.
As mentioned before, here we discuss the normality results of OCP, in the partic-ular of CVP. Therefore, we start by seeing the (CV P2) as a special case of (OCPM 2).
For that it is enough to consider a new absolutely continuous state variable
z(t) = Z t
0
L(x(s), ˙x(s))ds
and a change of variable ˙x(t) = u.
The (CV P2) can then be written as:
(OCPM 4) Minimize y(1)
subject to ˙z(t) = f (z(t), u(t)) a.e.t ∈ [0, 1] (x(0), y(0) = (x0, 0) u(t) ∈ Rn h(x(t)) ≤ 0 ∀t ∈ [0, 1] with z(t) = x(t) y(t) and f (z(t), u(t)) = u(t) L(x(t), u(t)) .
CQ ensuring normality of OCP with state constraints of the form h(x(t)) ≤ 0 are of two types:
CQ1n :∃˜u(t) ∈ Ω(t):a.e.t ∈ [0, 1]
ζ · [f (t, ¯x(t), ˜u(t)) − f (t, ¯x(t), ¯u(t))] < −δ,
for all ζ ∈ ∂>h(¯x(s)), when s ∈ {t ∈ [0, 1] : h(¯x(t)) = 0}, where ∂>h(x) is defined
as:
CQ2n: ∃ > 0 and ˜u(t) ∈ Ω(t):
hx(¯x(t)) · f (t, ¯x(t), ˜u(t)) < −δ,
for t ∈ (s − , s + ) where s ∈ {t ∈ [0, 1] : h(¯x(t)) = 0}. 1
In the work of [FV94], the normality of NCO for smooth CVP is guaranteed for problems that satisfied the following constraint qualification:
CQ4FV94 hx(¯x(t)) 6= 0 for t ∈ {s ∈ [0, 1] : h(¯x(s)) = 0}.
Two question arises:
• Since the work of [Fon00] allows possibly nonsmooth data for OCP, do we have strengthened NCO for CVP with nonsmooth data applying the normality result in [Fon00]?
• does the CQ of type CQ2n give new information when it is applied to CVP?
The answers to these questions are in next sections.
4.2
Normality in CVP Applying the Normal
Re-sult of [Fon00]
Applying the normal result of [Fon00] in CVP, we can obtain a strengthened NCO with nonsmooth data. This work was developed in [Lop03] and we mentione the result.
Assume that the following hypotheses are satisfied:
H1nCV The function x → L(x, u) is locally Lipschitz continuous for all u ∈ Rn.
H2nCV The function u → L(x, u) is convex and bounded for all x ∈ Rn.
1In [RV99], this CQ have to be satisfy on neighborhood of state constraint boundary, we not
H3nCV There exists an increasing function θ : [0, ∞) → [0, ∞) such that
lim
α→∞
θ(α)
α = +∞,
and a constant β such that L(x, v) > θ(kvk) − βkvk for all x ∈ Rn, v ∈ Rn. H4nCV There exists a scalar Kh > 0 such that the function x → h(x) is Lipschitz
continuous of rank Kh for all t ∈ [0, 1].
Consider also the following constraint qualifications:
CQCV1 If h(x0) = 0, then ∃ε0, δ > 0 such that
γ1· γ2 > δ,
∀γ1, γ2 ∈ ∂>h(x), and x ∈ {x0} + ε0B.
CQCV2 ∃δ > 0:
γ1· γ2 > δ,
∀γ1, γ2 ∈ ∂>h(¯x(s)), and s ∈ {t ∈ [0, 1] : h(¯x(t)) = 0}.
Proposition 4.2.1 Let (¯x, ¯u) be a local minimizer for (CV P2). Assume that
hy-potheses H1nCV− H4nCV and constraint qualifications CQCV1− CQCV2 are
sat-isfied. Then there exist p ∈ W1,1([0, 1] : Rn), a measurable function γ and a non-negative Radon measure µ ∈ C∗([0, 1], R) such that
˙
p(t) ∈ co ∂xL(L(¯x(t),x (t))) and q(t) ∈ co ∂¯. uL(L(¯x(t),x (t))),¯.
(p(0), −q(1)) ∈ NCL0×C1(¯x(0), ¯x(1)), γ(t) ∈ ∂x>h(¯x(t)) µ − a.e., supp {µ} ⊂ {t ∈ [0, 1] : h(¯x(t)) = 0},
where q(t) = p(t) + Z [0,t) γ(s)µ(ds) t ∈ [0, 1) p(t) + Z [0,1] γ(s)µ(ds) t = 1.
Remark 4.2.2 In the case when h is continuously differentiable, the set ∂>h(¯x(s))
is a singleton. Therefore, this CQCV2 reduce to hx(¯x(s)) 6= 0, confirming the CQ
in [FV94].
This proposition generalize the result of [FV94] by allowing nonsmooth data.
4.3
Normality in CVP Applying the Normal
Re-sult of [RV99] or [CF05]
To answer the question: “does the CQ of type CQ2n give new information when it
is applied to CVP?”, we apply the constraint qualification CQ2n to (OCPM 4).
So, we assume ∃˜u(t) ∈ Rn such that
hz(¯x) · f ((¯x, y), ˜u) < −δ, for a constant δ > 0. Consequently, (hx(¯x), 0) · ˜ u L(¯x, ˜u) < −δ.
Consider ˜u(t) = −hx(¯x(t)), we have hx(¯x) · (−hx(¯x)) = −khx(¯x)k2.
It follows that, for CVP, the constraint qualification CQ2n reduces to
hx(¯x) 6= 0.
Comparing this CQ with the CQCV1− CQCV2, we conclude that the latter is
In summary, we can say that in the case of OCP the NCO of [RV99] and [CF05] in comparison with the NCO of [Fon00] have the advantage that they do not involve the control function explicitly, and therefore are easier to verify.
However, in the special case of CVP the CQCV1− CQCV2, (obtained from the
results in [Fon00] for OCP) have the advantage that they apply to a wider class of problems.
Notes on Chapter
In [LF06], we can find a more detailed comparison between CQCV1− CQCV2 and
CQ obtained by applying the normality result of [RV99].
Part of the contents of section 4.2 have been presented in [Lop03] (see also [LF03]).
Normality of Optimal Control
Problems via Linearization of
Control Systems
The main objective of this chapter is to discuss normality of the MP for constrained problems with Lipschitz optimal trajectories. To prove normality, we use J. Yorke type linearization of control systems and show the existence of a solution to a lin-earized control system satisfying new state constraints. Our main result differers from similar results in the literature since we assume distinct set of hypothesis.
5.1
Introduction
In this chapter we consider the optimal control problem with implicit state con-straints: (OCP1) Minimize g(x(0), x(1))
subject to ˙x(t) = f (t, x(t), u(t)) a.e.t ∈ [0, 1] x(0) ∈ C0
x(t) ∈ X
u(t) ∈ Ω(t) a.e.t ∈ [0, 1]. 41
Remark 5.1.1 Define the signed distance by d(x) = −dist(x, bdy (X)) ∀x ∈ X, dist(x, bdy (X)) otherwise.
The problem (OCP1) is equivalent to replacing the state constraint (5.1) by the
inequality state constraint
d(x(t)) ≤ 0 for all t ∈ [0, 1].
Assume that Basic hypotheses H1b-H4b (see section 2.3.2) and the following
hypothesis is satisfied:
H5n Int CX(¯x(t)) 6= ∅ for each t ∈ [0, 1]. (This is a sufficient conditions for Vinter’s
CQ, see [Vin00].)
Here CC(x) denotes the Clarke’s tangent cone,
CC(x) = {v ∈ Rn| lim h→0+x0→
Cx
dist(x0+ hv, C)
h = 0}.
Then the Maximum Principle is the following:
Theorem 5.1.2 (The Maximum Principle for Implicit State Constraints)[Vin00] (Section 9.3) There exists an absolutely continuous function p : [0, 1] → Rn, η ∈
C∗([0, 1] : Rn), and λ ≥ 0 such that
Z
[0,1]
ζ(t) · η(dt) ≤ 0
for all ζ ∈ C([0, 1] : Rn) satisfying ζ(t) ∈ C
X(¯x(t)) η a.e.
(p, η, λ) 6= 0,
supp{η} ⊂ {t ∈ [0, 1] : ¯x(t) ∈ bdy (X)}, − ˙p(t) ∈ co ∂L
(p(0), −q(1)) ∈ λ∂Lg(¯x(0), ¯x(1)) + NCL0(¯x(0)) × {0}, H (t, ¯x(t), q(t), ¯u(t)) = max u∈Ω(t)H (t, ¯x(t), q(t), u) a.e.. Here q(t) = p(t) + Z [0,t) η(ds) t ∈ [0, 1) p(t) + Z [0,1] η(ds) t = 1.
In this chapter we assume a CQ to ensure the normality. This CQ is typically of the kind: there exists a control u and > 0 such that
dx(x) · f (t, x, u) ≤ −ρ for all x ∈ ¯x(t) + B, t ∈ [0, 1], ¯x(t) ∈ bdy (X) (5.2)
for some positive ρ.
Results ensuring normality using CQ of the type mention above can be found in [RV99], [CF05] and [BF07].
In this chapter we improve the result in [RV99], since we assume that the function defining by dynamics is merely measurable with respect to time.
In [RV99], the proof of the main result on normality is based on neighbouring feasible trajectories theorem. In this chapter, and also in [CF05] and [BF07], the proof is based in ensuring the existence of a solution to the problem
˙ w = γ(t, w) + ϕ(t), ϕ(t) ∈ Tco(f (t,¯x(t),Ω(t)))( . ¯ x (t)) satisfying w(t) ∈ Int(TX(¯x(t))) ∀t ∈ [0, 1].
Here TC(x) denotes Contingent Cone,
TC(x) = {v ∈ Rn| lim inf h→0+
dist(x + hv, C)
h = 0}.
k(·)-Lipschitz function, instead of k ∈ L∞ as it was proved in [CF05]. The result on normality of [BF07] allows the set X be nonsmooth, however the continuity of u → f (t, x, u) is assumed.
5.2
Normality Result via Linearization
Assume also the following hypotheses:
H6n Let X ⊂ Rn be closed and such that for some η > 0 the signed distance d(·)
is of class Cloc1,1 on bdy (X) + ηB. H7n Int (TX(¯x(0))) ∩ Int (TC0(¯x(0))) 6= ∅.
H8n There exist t0, t1, ..., tm such that 0 = t0 < t1 < t2... < tm = 1 and for all
i ∈ {0, ..., m − 1} either ¯x(ti, ti+1) ⊂ Int (X) and ¯x(ti), ¯x(ti+1) ∈ bdy (X) or
¯
x([ti, ti+1]) ⊂ bdy (X).
CQnVL Assume that for all R > 0, there exists r > 0 and ρ > 0 such that for a.e. t ∈ [0, 1] the following holds true
∀x ∈ (bdy (X) + ρB) ∩ RB
inf{dx(x) · f (t, x, u) : u ∈ Ω(t), kf (t, x, u)k ≤ r} ≤ −ρ.
We are now in position to state the main result of this chapter.
Theorem 5.2.1 Let (¯x, ¯u) be a local minimizer for the problem (OCP1), where ¯x
is Lipschitz continuous. Assume that the hypotheses H1b-H4b and H5n− H8n and
the constraint qualification CQnVL are satisfied. Then the MP for implicit state constraints theorem 5.1.2 holds true with λ = 1.
Remark 5.2.2 By the regularity hypotheses on the set X, we conclude that TX(¯x(t)) =
CX(¯x(t)), see [Cla83].
Lemma 5.2.3 Assume that the assumptions of the theorem and CQnV Lholds. Then
there exist 0 < δ < η, ρ > 0 and v(t) ∈ f (t, ¯x(t), Ω(t)) ∩ rB such that v(·) is measurable and
dx(¯x(t)) · v(t) ≤ −ρ,
whenever dist(¯x(t), bdy (X)) ≤ δ.
Lemma 5.2.4 Assume that there exist a function γ : [0, 1] × Rn → Rn measurable
in the first variable and for some k ∈ L1 and a.e. t ∈ [0, 1], γ(t, ·) is k(t)-Lipschitz.
The hypotheses H6n-H8n and the constraint qualification CQNVL are satisfied. Ad-ditionally assume that ¯x : [0, 1] → X is a Lipschitz function, the function whose exis-tence is assumed in Lemma (5.2.3) is essentially bounded and for that v ∈ L∞(0, 1), be such that for some ρ > 0 and a.e. t ∈ [0, 1] with ¯x(t) ∈ bdy (X) + ηB
dx(¯x(t)) · v(t) ≤ −ρ.
Then for every w0 ∈ Int (TX(¯x(0)) ∩ Int (TC0(¯x(0))) there exists an absolutely con-tinuous solution w to
˙
w = γ(t, w) + µ(t)(v(t)−x (t)), w(0) = w¯. 0 (5.3)
such that
w(t) ∈ Int TX(¯x(t)), (5.4)
for all t ∈ [0, 1], for some µ ∈ L1 such that µ(t) ≥ 0.
Remark 5.2.5 Define Γ = {t ∈ [0, 1] : ¯x(t) ∈ bdy (X) + ηB}. By the measurable selection theorem (see for instance 10.2.58 in appendix), there exists a measurable selection v(t) ∈ f (t, ¯x(t), Ω(t)) such that dx(¯x(t)) · v(t) ≤ −ρ for almost all t ∈ Γ.
We extend v on [0, 1] by setting v(t) =x (t) for all t 6∈ Γ. Then µ(t)(v(t)−¯. x (t)) ∈¯. Tco(f (t,¯x(t),Ω(t)))(
.
¯ x (t)).
