CONVEX COMPUTATION OF THE MAXIMUM CONTROLLED
INVARIANT SET FOR POLYNOMIAL CONTROL SYSTEMS∗
MILAN KORDA†, DIDIER HENRION‡, AND COLIN N. JONES†
Abstract. We characterize the maximum controlled invariant (MCI) set for discrete- as well as continuous-time nonlinear dynamical systems as the solution of an infinite-dimensional linear programming problem. For systems with polynomial dynamics and compact semialgebraic state and control constraints, we describe a hierarchy of finite-dimensional linear matrix inequality (LMI) relaxations whose optimal values converge to the volume of the MCI set; dual to these LMI relaxations are sum-of-squares (SOS) problems providing a converging sequence of outer approximations to the MCI set. The approach is simple and readily applicable in the sense that the approximations are the outcome of a single semidefinite program with no additional input apart from the problem description. A number of numerical examples illustrate the approach.
Key words. maximum controlled invariant set, region of attraction, optimal control, nonlinear control, sum of squares, occupation measure, moments, semidefinite programming
AMS subject classifications.49M, 49L, 90C22
DOI.10.1137/130914565
1. Introduction. Given a controlled dynamical system described by a differen-tial (continuous-time) or difference (discrete-time) equation, its maximum controlled invariant (MCI) set is the set of all initial states that can be kept within a given con-straint set ad infinitum using admissible control inputs. This set goes by many other names in the literature, e.g., viability kernel in viability theory [5], or (A, B)-invariant set in the linear case [13].
Set invariance is a ubiquitous and essential concept in dynamical systems theory as far as both analysis and control synthesis is concerned. In particular, by its very definition, the MCI set determines fundamental limitations of a given control system with respect to constraint satisfaction. In addition, there is a very tight link between invariant sets and (control) Lyapunov functions. Indeed, sublevel sets of a Lyapunov function give rise to invariant sets. Conversely, at least in the linear case, any con-trolled invariant set gives rise to a control Lyapunov function, and therefore these sets can be readily used to design stabilizing control laws; see, e.g., [9] for a general treatment and, e.g., [17, 26] for applications in model predictive control design.
The problem of (maximum) controlled invariant set computation for discrete-time systems has been a topic of active research for more than four decades. The central tool in this effort has been the contractive algorithm of [7] and its expansive counterpart [18]. For an exhaustive survey and historical remarks see the survey [9] and the book [12].
∗Received by the editors March 26, 2013; accepted for publication (in revised form) June 11, 2014;
published electronically September 25, 2014.
http://www.siam.org/journals/sicon/52-5/91456.html
†Laboratoire d’Automatique, ´Ecole Polytechnique F´ed´erale de Lausanne, Station 9, CH-1015,
Lausanne, Switzerland (milan.korda@epfl.ch, colin.jones@epfl.ch).
‡CNRS, LAAS, 7 avenue du colonel Roche, F-31400 Toulouse, France ([email protected]), Universit´e
de Toulouse, LAAS, F-31400 Toulouse, France, and the Faculty of Electrical Engineering, Czech Technical University in Prague, Technick´a 2, CZ-16626 Prague, Czech Republic. This author’s research was partly supported by project 13-06894S of the Grant Agency of the Czech Republic. A preliminary version of this work, dealing with discrete-time systems only, was presented at the IEEE Conference on Decision and Control, 2013.
Both algorithms, although conceptually applicable to any nonlinear system, have been predominantly applied in a linear setting where they boil down to a sequence of linear programs (LPs) and polyhedral projections. Finite termination of this sequence is a subtle problem, and sharp results are available only in the uncontrolled setting where no projections are required [16]; for discussion of finite-termination in the con-trolled case see [45]. The contractive and expansive algorithms were combined in [17] to design an algorithm terminating in a finite number of iterations and outputting an -accurate inner approximation of the MCI set (with the accuracy measured by the Hausdorff distance). Another line of research, culminating in [42], exploits the linear-ity of the system dynamics in a more systematic way and approximates the maximum (or minimum)robustcontrolled invariant set by the Minkowski sum of a parametrized family of sets. Very recently, in continuous time, [30] developed a parallel algorithm for ellipsoidal approximations of the robust MCI set scalable to very high dimensions. Computation of low-complexity polyhedral controlled invariant sets was investigated in [10] and [11].
In the nonlinear case, a common practice is to exploit the tight connection be-tween invariance and Lyapunov functions and seek invariant sets as sublevel sets of a (control) Lyapunov function; see, e.g., [14, 51] and references therein for recent theoretical developments on the related problem of region of attraction (ROA) com-putation and, e.g., [34] for practical applications of these techniques. This, however, typically leads to nonconvex bilinear optimization problems which are notoriously hard to solve. Therefore, one often has to resort to ad hoc analysis of the specific system at hand, which is typically tractable only in small dimensions; see [46, 47] for concrete examples. Related in spirit is the localization technique of [25] for
discrete-timeuncontrolled systems, also requiring considerable effort in analyzing the system.
Recently, a general approach using a hierarchy of finite-dimensional LPs was used in [6] to design a controller ensuring invariance of a given candidate polyhedral set. In our opinion, although being the current state of the art, this work still suffers from the following drawbacks: (1) the sets obtained are convex polytopes (not general semialgebraic sets, a fact particularly limiting in the nonlinear case where nonconvex MCI sets are common); (2) the geometry of the candidate polytopic set must be given a priori; and (3) there are no convergence guarantees to the MCI set. In this paper, we explicitly address all these points.
Building upon our previous work [19] on the computation of the ROA for polyno-mial control systems, in this paper we characterize the MCI set for discrete- as well as continuous-time polynomial systems as the solution to an infinite-dimensional linear programming problem in the cone of nonnegative measures. The dual of this problem is an infinite-dimensional LP in the space of continuous functions. Finite-dimensional
relaxations of the primal LP and finite-dimensional approximations of the dual LP
turn out to be semidefinite programs (SDPs) also related by duality. The primal re-laxations lead to a truncated moment problem, while the dual approximations lead to a sum-of-squares (SOS) problem. Superlevel sets of one of the polynomials appearing in the dual SOS problem then provide outer approximations to the MCI set with guaranteed convergence as the degree of the polynomial tends to infinity.
The main mathematical tool we use are the so-called occupation measures, which allow us to study the time evolution of the whole ensemble of initial conditions (de-scribed by a measure) rather than studying trajectories associated to each initial condition separately. The use of measures to study dynamical systems has a very
long tradition: see [44] for probably the first systematic treatment;1 for a purely discrete-time treatment see [22, Chapter 6]. To the best of the authors’ knowledge our paper is the first to use occupation measures for MCI set (approximate)
compu-tation. The MCI set was previouslycharacterized using occupation measures in [15],
but there the characterization is rather indirect and not straightforwardly amenable to computation. Apart from the authors’ work [19], the related problem of ROA computation was tackled using measures in [52]. There, however, a very different approach was taken, not usingoccupation measures but rather analyzing convergence via discretization of the state-space and propagating the initial distribution by means of a discretized transfer operator. Here, instead, we employ the (discounted) occu-pation measure, which captures the behavior of the trajectories emanating from the initial distribution over the infinite time horizon. As a result, our approach requires no discretization and, contrary to [52], provides true guarantees (not in an “almost-everywhere” or “coarse” sense) and, more importantly, is applicable in a controlled setting. Closely related to the occupation measures used here is the Rantzer den-sity [43], which was used in [41] to assess the stability of attractor sets of uncontrolled nonlinear systems. That approach, however, does not immediately yield approxima-tions of the MCI set (or the ROA) and applies to uncontrolled systems only.
Similar in spirit to our approach, from the dual viewpoint of optimization over functions, are the Hamilton–Jacobi approaches (e.g., [35, 36]). However, contrary to these methods, our approach does not require state-space discretization and comes with convergence guarantees.
The contribution of our paper with respect to previous work on the topic can be summarized as follows:
• We deal with fully general continuous-time and discrete-time polynomial dy-namics under semialgebraic state and control constraints.
• Our approximated MCI set is described by (the intersections of) polynomial superlevel sets, including more restrictive classes (e.g., polytopes and ellip-soids).
• We provide a convex infinite-dimensional linear programming characteriza-tion of the MCI set.
• We describe a hierarchy of convex finite-dimensional SDPs to solve the LP with convergence guarantees.
• Our approach is simple and readily applicable in the sense that the approxi-mations are the result of a single SDP withno additional data required apart from the problem description.
The contribution with respect to our previous work [19] can be summarized as follows:
• In [19] we compute the ROA, which is a related although different object: it is the set of all of initial conditions that can be steered to a given target set while satisfying state and control constraints. In particular, the MCI set differs from the ROA in the sense that we do not try to hit any target set at a given time but rather try to keep the state within a given set forever. Therefore, we had to adapt our technique to deal explicitly with invariance;
• In [19] we dealt with continuous-time systems only, whereas here we address (with minor modifications) discrete-time systems as well; we choose to de-scribe both the continuous-time and discrete-time setups in parallel precisely
1In [44], J. E. Rubio used Young measures [50] rather than occupation measures, but the basic
idea of “linearizing” a nonlinear problem by going into an infinite-dimensional space of measures is the same.
to underline these common features.
• In [19] we considered ROA on a finite time-horizon, whereas here we char-acterize the MCI set, which is an inherently infinite-time object. This brought about additional technical difficulties not encountered on a finite time-horizon. What can be considered a drawback of our approach is the fact that the approx-imations to the MCI set we obtain are from the outside and therefore not invariant. However, accurate outer approximations provide important information as to the performance limitations of the control system and are of practical interest, e.g., in collision avoidance. Therefore, we believe that our work bears both theoretical and practical value and naturally complements existing inner-approximation techniques.
The paper is organized as follows. The problem to be solved is described in sec-tion 2. Occupasec-tion measures are introduced in secsec-tion 3. The infinite-dimensional primal and dual LPs are described in sections 4 and 5, respectively. The finite-dimensional relaxations with convergence results are presented in section 6. Numerical examples are in section 7. A reader interested only in the semialgebraic outer approx-imations of the MCI set can directly consult the infinite-dimensional dual LPs (8) and (9) and their finite-dimensional approximations (11) and (13) in discrete and continuous time, respectively.
1.1. Notation. Measures are understood as signed Borel measures on a Eu-clidean space, i.e., as countably additive maps from the Borel sets to the real num-bers. From now on all subsets of a Euclidean space we refer to are automatically understood as Borel. The vector space of all signed Borel measures with its support contained in a setX is denoted byM(X). The support (i.e., the smallest closed set whose complement has a zero measure) of a measureμis denoted by sptμ. The space of continuous functions onX is denoted byC(X), and likewise the space of once con-tinuously differentiable functions isC1(X). The indicator function of a setX (i.e., a function equal to one onXand zero otherwise) is denoted byIX(·). The symbolλ de-notes then-dimensional Lebesgue measure (i.e., the standardn-dimensional volume). The integral of a function v with respect to a measureμover a set X is denoted by
Xv(x)dμ(x). Sometimes for conciseness we use the shorter notation
v dμ, omitting the integration variable and also the set over which we integrate if they are obvious from the context. The ring of polynomials in (possibly vector) variablesx1,. . . ,xn is denoted byR[x1, . . . , xn].
2. Problem statement. The approach is developed in parallel for discrete and continuous time.
2.1. Discrete time. Consider the discrete-time control system (1) xt+1 =f(xt, ut), xt∈X, ut∈U, t∈ {0,1, . . .},
with a given polynomial vector field f with entries fi ∈ R[x, u], i = 1, . . . , n, and given compact2basic semialgebraic state and input constraints
xt∈X :={x∈Rn : gX i(x)≥0, i= 1,2, . . . , nX}, ut∈U :={u∈Rm : g
U i(u)≥0, i= 1,2, . . . , nU}
2The assumption of compactness is crucial for the theoretical developments in the paper. The
assumptions that the constraint sets are basic semialgebraic and the mappingf(·,·) polynomial plays no role in the infinite-dimensional considerations and only facilitates the convergence results of the finite-dimensional relaxations. The above is valid for both discrete and continuous time.
withgX i∈R[x],gU i∈R[u].
The maximum controlled invariant (MCI) set is defined as XI := x0∈X :∃{xt}∞t=1,{ut}∞t=1 s.t. xt+1=f(xt, ut), ut∈U, xt∈X∀t∈ {0,1, . . .} .
A control sequence{ut}∞t=0 is calledadmissible ifut∈U for allt∈ {0,1, . . .}. In words, the MCI set is the set of all initial states which can be kept inside the constraint setX ad infinitum using admissible control inputs.
2.2. Continuous time. Consider the relaxed continuous-time control system (2) x(t)˙ ∈convf(x(t), U), x(t)∈X, t∈[0,∞),
where conv denotes the convex hull, f is a polynomial vector field with entries fi ∈ R[x, u],i= 1, . . . , n,and compact basic semialgebraic state and input constraint sets are defined by
X :={x∈Rn : g
X i(x)≥0, i= 1,2, . . . , nX},
U :={u∈Rm : g
U i(u)≥0, i= 1,2, . . . , nU}
with gX i ∈R[x], gU i ∈R[u]. The meaning of the convex differential inclusion (2) is as follows: for all time t, the state velocity ˙x(t) is constrained to the convex hull of the set f(x(t), U) := {f(x(t), u) : u ∈ U} ⊂ Rn. The connection of this convexi-fied (or relaxed) control problem (2) and the classical control problem ˙x=f(x, u) is the Filippov–Wa ˙zewski theorem [5], which shows that the trajectories of ˙x=f(x, u) are dense (in the supremum norm) in the set of trajectories of the convexified in-clusion3 (2). Therefore, from a practical point of view, there is little difference be-tween the two formulations for the purposes of MCI set computation; see section 3.2 and Appendices B and C of [19] for a detailed discussion on this subtle issue. The simplest assumption under which the MCI sets for both systems coincide is f(x, U) being convex for allx, which is particularly true for input-affine systems of the form
˙
x=f(x) +g(x)uwithU convex. The MCI set is defined as XI :=
x0∈X : ∃x(·) s.t. x(t)˙ ∈convf(x(t), U) a.e., x(t)∈X∀t∈[0,∞)
, wherex(·) is required to be absolutely continuous and a.e. stands for “almost every-where” with respect to the Lebesgue measure on [0,∞).
In words, the MCI set is the set of all initial states for which there exists a trajectory of the convexified inclusion (2) which remains inX ad infinitum.
3. Occupation measures. In this section we introduce the concept of occupa-tion measures, which is the centerpiece of our approach.
3Note that the set convf(x(t), U) is closed for everytsincefis continuous andUis compact;
therefore, there is no need to take closure of the convex hull in order to apply the Filippov–Wa˙zewski theorem.
3.1. Discrete time. Given a discount factorα∈(0,1), an initial conditionx0, and an admissible control sequence{ut|x0}∞t=0such that the associated state sequence
{xt|x0}∞t=0 remains in X for all time, we define the discounted occupation measure
μ(· |x0)∈M(X×U) as (3) μ(A×B|x0) := ∞ t=0 αtIA×B(xt|x0, ut|x0)
for all setsA⊂X andB⊂U.
In words, the discounted occupation measure measures the (discounted) number of visits of the state-control pair trajectory (x(·|x0), ν(·|x0)) to subsets ofX×U. The discounting in the definition of the occupation measure ensures thatμ(A×B|x0) is always finite; in fact, we haveμ(X×U |x0) = (1−α)−1.
Now suppose that the initial condition is not a single point but aninitial measure4
μ0∈M(X) and an admissible control sequence is associated to each initial condition from the support ofμ0 in such a way that the corresponding state sequence remains inX. Then we define theaverage discounted occupation measure μ∈M(X×U) as
μ(A×B) :=
X
μ(A×B|x0)dμ0(x0).
The average discounted occupation measure measures the discounted average number of visits in subsets of X ×U of trajectories starting from the initial dis-tributionμ0.
Now we derive an equation linking the measures μ0 and μ. This equation will play a key role in subsequent development and in a sense replaces the dynamics equation (1). To derive this equation, fix an initial conditionx0 ∈X and a control sequence {ut|x0}∞t=0 such that the associated state sequence {xt|x0}∞t=0 stays in X. Then for anyv∈C(X) we have
X×U v(x)dμ(x, u|x0) = ∞ t=0 αtv(xt|x0) =v(x0|x0) +α ∞ t=0 αtv(xt+1|x0) =v(x0|x0) +α ∞ t=0 αtv(f(xt|x0, ut|x0)) =v(x0|x0) +α X×U v(f(x, u))dμ(x, u|x0). Integrating with respect toμ0, we arrive at the sought equation (4) X×U v(x)dμ(x, u) = X v(x)dμ0(x) +α X×U v(f(x, u))dμ(x, u) ∀v∈C(X). Note that this is an infinite-dimensional linear equation in variables (μ0, μ). The following crucial lemma establishes the connection between the support of any initial measureμ0solving (4) and the MCI setXI.
Lemma 1. For any pair of measures (μ0, μ) satisfying (4) with sptμ0 ⊂X and
sptμ⊂U ×X we have sptμ0⊂XI.
Proof. A detailed proof is in Appendix A.
4The initial measureµ0 can be thought of as the probability distribution of the initial state,
3.2. Continuous time. Given an initial conditionx0 and a trajectoryx(· |x0) of the inclusion (2) that remains in X for allt≥0, there exists an admissible time-varying measure-valued relaxed controlνt(·|x0)∈M(U),νt(U|x0) = 1, such that
˙ x(t) =
U
f(x(t), u)dνt(u|x0)
a.e. with respect to the Lebesgue measure on [0,∞). This follows from the definition of the convex hull (in fact, for eacht,νt(·|x0) can be taken to be a convex combination of finitely many Dirac measures).
Then, given a discount factorβ >0, we define thediscounted occupation measure
μ(· |x0)∈M(X×U) as μ(A×B |x0) := ∞ 0 U e−βtIA×B(x(t|x0), u)dνt(u|x0)dt for all setsA⊂X andB⊂U.
In words, the discounted occupation measure measures the (discounted) time spent by the state-control pair trajectory (x(·|x0), ν(·|x0)) in subsets ofX×U. The discounting in the definition of the occupation measure ensures thatμ(A×B|x0) is always finite; in fact, we haveμ(X×U |x0) =β−1.
Now suppose that the initial condition is not a single point but aninitial measure5
μ0∈M(X) and a state trajectory that remains inXalong with an admissible relaxed control is associated to each initial condition from the support ofμ0. Then we define
theaverage discounted occupation measure μ∈M(X×U) as
μ(A×B) :=
X
μ(A×B|x0)dμ0(x0).
Now we derive an equation linking the measures μ0 and μ. This equation will play a key role in subsequent development and in a sense replaces the dynamics equation (2). To derive the equation, fix an initial conditionx0∈X and a trajectory x(· |x0) that remains in X with an associated admissible relaxed control νt(· |x0). Then for anyv∈C1(X) integration by parts yields
X×U gradv·f(x, u)dμ(x, u|x0) = ∞ 0 U e−βtgradv·f(x(t|x0), u)dνt(u|x0)dt = ∞ 0 e−βtd dtv(x(t|x0))dt =β ∞ 0 e−βtv(x(t|x0))dt−v(x(0|x0)) =β X×U v(x)dμ(x, u|x0)−v(x(0|x0)),
where the boundary term at infinity vanishes due to discounting and the fact thatX is bounded. Integrating with respect to μ0then gives the sought equation
(5) β X×U v(x)dμ(x, u) = X v(x)dμ0(x) + X×U gradv·f(x, u)dμ(x, u) ∀v∈C1(X).
5The initial measureµ0 can be thought of as the probability distribution of the initial state,
Note that this is an infinite-dimensional linear equation in variables (μ0, μ). The following crucial lemma establishes the connection between the support of any initial measure satisfying (5) and the MCI setXI.
Lemma 2. For any pair of measures (μ0, μ) satisfying (5) with sptμ0 ⊂X and
sptμ⊂U ×X we have λ(sptμ0)≤λ(XI).
Proof. A detailed proof is in Appendix B.
4. Primal LP. In this section we show how the MCI set computation problem can be cast as an infinite-dimensional linear programming problem in the cone of nonnegative measures. As in [19], the basic idea is to maximize the mass of the initial measure μ0 subject to the constraint that it be dominated by the Lebesgue measure, that is, μ0 ≤λ. System dynamics is captured by (4) and (5) for discrete and continuous times, respectively; state and input constraints are expressed through constraints on the supports of the initial and occupation measures. The constraint that μ0 ≤ λ can be equivalently rewritten as μ0 + ˆμ0 = λ for some nonnegative slack measure ˆμ0∈M(X). This constraint is in turn equivalent toXw(x)dμ0(x) +
Xw(x)dˆμ0(x) =
Xw(x)dλ(x) for allw∈C(X). These considerations lead to the
following primal LPs.
4.1. Discrete time. The primal LP in discrete time reads (6) p∗ = sup μ0(X) s.t. v(x)dμ(x, u) =v(x)dμ0(x) +αv(f(x, u))dμ(x, u)∀v∈C(X), w(x)dμ0(x) +w(x)dˆμ0(x) =w(x)dλ(x) ∀w∈C(X), μ≥0, μ0≥0, μˆ0≥0, sptμ⊂X×U, sptμ0⊂X, spt ˆμ0⊂X,
where the supremum is over the vector of measures (μ, μ0,μˆ0)∈M(X×U)×M(X)× M(X).
This is an infinite-dimensional LP in the cone of nonnegative Borel measures. The following theorem, which is our main theoretical result, relates an optimal solution of this LP to the MCI setXI.
Theorem 1. The optimal value of linear programming problem (6)is equal to the
volume of the MCI set XI, that is, p∗=λ(XI). Moreover, the supremum is attained
by the restriction of the Lebesgue measure to the MCI set XI.
Proof. The proof follows from Lemma 1 by the same arguments as Theorem 1 in
[19]. By definition of the MCI set XI, for any initial conditionx0∈XI there exists an admissible control sequence such that the associated state sequence remains inX. Therefore, for any initial measureμ0≤λwith sptμ0 ⊂XI there exist a discounted occupation measureμ with sptμ⊂X×U and a slack measure ˆμ0 with spt ˆμ0⊂X such that the constraints of (6) are satisfied. One such measureμ0is the restriction of the Lebesgue measure toXI, and thereforep∗≥λ(XI). The factp∗≤λ(XI) follows from Lemma 1.
4.2. Continuous time. The primal LP in continuous time reads
(7) p∗ = sup μ0(X) s.t. βv(x)dμ(x, u) =v(x)dμ0(x) + gradv·f(x, u)dμ(x, u)∀v∈C1(X), w(x)dμ0(x) + w(x)dμˆ0(x) = w(x)dλ(x) ∀w∈C(X), μ≥0, μ0≥0,μˆ0≥0, sptμ⊂X×U, sptμ0⊂X, spt ˆμ0⊂X,
where the infimum is over the vector of measures (μ, μ0,μˆ0)∈M(X×U)×M(X)× M(X).
This is an infinite-dimensional LP in the cone of nonnegative Borel measures. The following theorem, which is our main theoretical result, relates an optimal solution of this LP to the MCI setXI.
Theorem 2. The optimal value of LP problem (7) is equal to the volume of
the MCI set XI, that is, p∗ = λ(XI). Moreover, the supremum is attained by the
restriction of the Lebesgue measure to the MCI setXI.
Proof. The fact that μ0 is equal to the restriction of the Lebesgue measure to
XI is feasible in (7) (and therefore p∗≥λ(XI)) follows by the same arguments as in discrete time. The fact thatp∗≤λ(XI) follows from Lemma 2.
5. Dual LP. In this section we derive LPs dual to the primal LPs (6) and (7). Since the primal LPs are in the space of measures, the dual LPs will be on the space of continuous functions. Superlevel sets of feasible solutions to these LPs then provide outer approximations to the MCI sets, both in discrete and in continuous time. Both duals can be derived by standard infinite-dimensional linear programming duality theory; see [19] for a derivation in a similar setting or [3] for a general theory of infinite-dimensional linear programming.
5.1. Discrete time. The dual LP in discrete time reads
(8) d∗ = inf X w(x)dλ(x) s.t. αv(f(x, u))≤v(x) ∀(x, u)∈X×U, w(x)≥v(x) + 1 ∀x∈X, w(x)≥0 ∀x∈X,
where the infimum is over the pair of functions (v, w)∈C(X)×C(X).
The following key observation shows that the unit superlevel set of any function wfeasible in (8) provides an outer approximation toXI.
Lemma 3. Any feasible solution to (8) satisfiesv≥0 andw≥1 on XI.
Proof. Given anyx0 ∈ XI, there exists a sequence {ut}∞t=0, ut ∈ U, such that
xt∈X for allt. The first constraint of problem (8) is equivalent toαv(xt+1)≤v(xt), t∈ {0,1, . . .}. By iterating this inequality we get
v(x0)≥αtv(xt)→0 as t→ ∞
since xt ∈ X and X is bounded. Therefore, v(x0)≥ 0 andw(x0)≥ 1 for all x0 ∈ XI.
The following theorem is instrumental in proving the convergence results of sec-tion 6.
Theorem 3. There is no duality gap between primal linear programming problem
(6) on measures and dual linear programming problem (8) on functions in the sense
that p∗=d∗.
Proof. The proof follows by the same arguments as does Theorem 2 in [19] using
standard infinite-dimensional linear programming duality theory (see, e.g., [3]) and the fact that the feasible set of the primal LP is nonempty and bounded in the metric inducing the weak-* topology onM(X)×M(X×U)×M(X). To see nonemptiness, notice that the vector of measures (μ0, μ,μˆ0) = (0,0, λ) is trivially feasible. To see the boundedness, it suffices to evaluate the equality constraints of (6) forv(x) =w(x) = 1. This gives μ0(X) + ˆμ0(X) = λ(X) <∞ and μ(X) = μ0(X)/(1−α), which, since α∈(0,1) and all measures are nonnegative, proves the assertion.
5.2. Continuous time. The dual LP in continuous time reads (9) d∗ = inf X w(x)dλ(x) s.t. gradv·f(x, u)≤βv(x) ∀(x, u)∈X×U, w(x)≥v(x) + 1 ∀x∈X, w(x)≥0 ∀x∈X,
where the infimum is over the pair of functions (v, w)∈C1(X)×C(X).
The following key observation shows that the unit superlevel set of any function wfeasible in (9) provides an outer approximation toXI.
Lemma 4. Any feasible solution to (9) satisfiesv≥0 andw≥1 on XI.
Proof. Given anyx0 ∈ XI, there exists an admissible relaxed control function
νt(·), νt(U) = 1, such that x(t) ∈ X for all t. For that x(t) we have dtdv(x(t)) =
Ugradv·f(x(t), u)dνt(u)≤
Uβv(x(t))dνt(u) = νt(U)βv(x(t)) = βv(x(t)). Then
by Gronwall’s inequalityv(x(t))≤eβtv(x0), and consequently v(x0)≥e−βtv(x(t))→0 as t→ ∞
since x(t) ∈ X and X is bounded. Therefore, v(x0) ≥ 0 and w(x0) ≥ 1 for all x0∈XI.
The following theorem is instrumental in proving the convergence results of sec-tion 6.
Theorem 4. There is no duality gap between primal linear programming problem
(7) on measures and dual linear programming problem (9) on functions in the sense
that p∗=d∗.
Proof. The proof follows by the same arguments as does Theorem 2 in [19] using
standard infinite-dimensional linear programming duality theory (see, e.g., [3]) and the fact that the feasible set of the primal LP is nonempty and bounded in the metric inducing the weak-* topology onM(X)×M(X×U)×M(X). To see nonemptiness, notice that the vector of measures (μ0, μ,μˆ0) = (0,0, λ) is trivially feasible. To see the boundedness, it suffices to evaluate the equality constraints of (7) forv(x) =w(x) = 1. This givesμ0(X) + ˆμ0(X) =λ(X) <∞ and μ(X) = μ0(X)/β, which, since β > 0 and all measures are nonnegative, proves the assertion.
6. SDP relaxations. In this section we present finite-dimensional relaxations/ approximations of the infinite-dimensional primal/dual LPs. Both in continuous and discrete time, therelaxations of the primal LPs lead to a truncated moment problem which translates to an SDP that can be solved by freely available software, e.g., SeDuMi [39] or SDPA [48]. Dual to the primal SDP relaxation is an SOS problem that again translates to an SDP problem; this dual SDP problem can be interpreted as anapproximation of the dual infinite-dimensional LPs in the sense that the space of functions searched over is restricted to polynomials of bounded degrees.
The following discussion, adapted from [27], highlights the main ideas behind the derivation of the finite-dimensional relaxations/approximations. The reader is referred to [19, section 5] or to the comprehensive reference [31] for details. First, since the supports of all measures feasible in (6) and (7) are compact, these measures are uniquely determined by their moments, i.e., by integrals of all monomials (which is a sequence of real numbers when indexed in, e.g., the canonical monomial basis). Therefore, it suffices to restrict the test functionsw(x) andv(x) in (6) and (7) to all monomials, reducing the linear equality constraints on measuresμ0, μ, and ˆμ0of (6)
and (7) to linear equality constraints on their moments. Next, by the Putinar Posi-tivstellensatz (see [31, 40]), the constraint that the support of a measure be included in a given compact basic semialgebraic set is equivalent (modulo a nonrestrictive al-gebraic condition; see Assumption 1) to the feasibility of an infinite sequence of linear matrix inequalities (LMIs) involving the so-called moment and localizing matrices, which are linear in the coefficients of the moment sequence. By truncating the mo-ment sequence and taking only the momo-ments corresponding to monomials of total degree less than or equal to 2k, wherek∈ {1,2, . . .}is the relaxation order, we obtain a necessary condition for this truncated moment sequence to be the first part of a moment sequence corresponding to a measure with the desired support.
Notation. In what follows,Rk[·] denotes the vector space of real multivariate
poly-nomials of total degree less than or equal tok. Further, we letdX i:=deg(gX i)/2 and dU i := deg(gU i)/2, where deg(·) denotes the degree of a polynomial; we also use0 to denote positive semidefiniteness of a matrix. In the primal SDP relaxations we use the notion of moment and localizing matricesMk(·) andMk(·,·), following the notation of [31] or [19, section 6]. Finally, throughout the rest of this section we make the following standard assumption.
Assumption 1. One of the polynomials modeling the sets X, respectively, U, is
equal to gX i(x) =R2X− x22, respectively, gU i(u) =R2U− u22, for some RX ≥0,
RU ≥0.
This assumption is made completely without loss of generality since redundant ball constraints can always be added to the description of thecompact setsX andU.
6.1. Discrete time. The primal relaxation of orderkin discrete time reads (10) p∗k = max (y0)0 s.t. Ak(y, y0,yˆ0) =bk, Mk(y)0, Mk−dX i(gX i, y)0, i= 1,2, . . . , nX, Mk−dU i(gU i, y)0, i= 1,2, . . . , nU, Mk(y0)0, Mk−dX i(gX i, y0)0, i= 1,2, . . . , nX, Mk(ˆy0)0, Mk−dX i(gX i,yˆ0)0, i= 1,2, . . . , nX, where the maximum is over moment sequences (y, y0,yˆ0) truncated to degree 2k cor-responding to measuresμ,μ0, and ˆμ0in (6). The linear equality constraint captures the two linear equality constraints of (6) withv(x) ∈R2k[x] and w(x)∈ R2k[x] be-ing monomials of total degree less than or equal to 2k. In (10), a linear objective is minimized subject to linear equality constraints and LMI constraints; therefore, (10) is an SDP.
The dual approximation of orderkin discrete time reads
(11) d∗k = inf wl s.t. v(x)−αv(f(x, u)) =q0(x, u) + nX i=1qi(x, u)gX i(x) +i=1nU ri(x, u)gU i(u), w(x)−v(x)−1 =p0(x) + nX i=1pi(x)gX i(x), w(x) =s0(x) + nX i=1si(x)gXi(x),
wherel is the vector of Lebesgue moments overX indexed in the same basis in which the polynomialw(x) with coefficientsw is expressed. The infimum (which may not be attained) is over polynomialsv(x)∈R2k[x] andw∈R2k[x], and polynomial SOS qi,pi,si,i= 1, . . . , nX, andri,i= 1, . . . , nU, of appropriate degrees. In (11), a linear objective function is minimized subject to SOS constraints; therefore, (11) is an SOS problem which can be readily cast as an SDP (see, e.g., [31]).
6.2. Continuous time. The primal relaxation of order k in continuous time reads (12) p∗k = max (y0)0 s.t. Ak(y, y0,yˆ0) =bk, Mk(y)0, Mk−dX i(gX i, y)0, i= 1,2, . . . , nX, Mk−dU i(gU i, y)0, i= 1,2, . . . , nU, Mk(y0)0, Mk−dX i(gX i, y0)0, i= 1,2, . . . , nX, Mk(ˆy0)0, Mk−dX i(gX i,yˆ0)0, i= 1,2, . . . , nX, where the maximum is over moment sequences (y, y0,yˆ0) truncated to degree 2k cor-responding to measuresμ,μ0, and ˆμ0in (7). The linear equality constraint captures the two linear equality constraints of (7) withv(x) ∈R2k[x] and w(x)∈ R2k[x] be-ing monomials of total degree less than or equal to 2k. In (12), a linear objective is minimized subject to linear equality constraints and LMI constraints; therefore, (12) is an SDP.
The dual approximation of orderkin continuous time reads
(13) d∗k = inf wl s.t. βv(x)−gradv·f(x, u) =q0(x, u)+ nX i=1qi(x, u)gX i(x)+i=1nU ri(x, u)gU i(u), w(x)−v(x)−1 =p0(x) + nX i=1pi(x)gX i(x), w(x) =s0(x) + nX i=1si(x)gXi(x),
wherel is the vector of Lebesgue moments overX indexed in the same basis in which the polynomialw(x) with coefficientsw is expressed. The infimum (which may not be attained) is over polynomialsv(x)∈R2k[x] andw∈R2k[x], and polynomial SOS qi,pi,si,i= 1, . . . , nX, andri,i= 1, . . . , nU, of appropriate degrees. In (13), a linear objective function is minimized subject to SOS constraints; therefore, (13) is an SOS problem which can be readily cast as an SDP (see, e.g., [31]).
6.3. Convergence results. In this section we state several convergence results for the finite-dimensional relaxations, respectively, approximations, (10), (12), re-spectively, (11), (13). Letwk andvk denote an optimal solution to thekth dual SDP approximation (11) or (13), and define
XI k:={x∈X : vk(x)≥0}.
Then, in view of Lemmas 3 and 4, we know thatwk overapproximates the indicator function of the MCI set XI on X, i.e., wk ≥ IXI on X, and that the sets XI k approximate from outside the MCI setXI, i.e.,XI k⊃XI. In what follows we prove the following:
• The optimal values of the finite-dimensional primal and dual problems p∗k andd∗kcoincide and converge to the optimal values of the infinite-dimensional primal and dual LPs p∗ and d∗ which also coincide (in view of Theorems 3 and 4) and are equal to the volume of the MCI set.
• The sequence of functions wk converges on X from above to the indicator function of the MCI set in theL1 norm. In addition, the running minimum mini≤kwi converges onX from above to the indicator function of the MCI set in theL1norm and almost uniformly.
• The sequence of setsXI k converges to the MCI setXI in the sense that the volume discrepancy tends to zero, i.e., limk→∞λ(XIk\XI) = 0.
The proofs of the results follow very similar reasoning to that of analogous results on ROA approximations in [19, section 6].
Lemma 5. There is no duality gap between primal LMI problems (10)and (12)
and dual LMI problems (11)and(13); i.e.,p∗k=d∗k.
Proof. The argument closely follows that in [19, Theorem 4], and therefore we
only outline the key points of the proof. To prove the absence of duality gap, it is sufficient to show that the feasible sets of the primal SDPs (10) and (12) are nonempty and compact. The result then follows by standard SDP duality theory (see [19, Theorem 4] for a detailed argument). The nonemptiness follows trivially since the vector of measures (μ0, μ,μ) = (0,ˆ 0, λ) is feasible in the primal infinite-dimensional LPs (6) and (7), and therefore the truncated moment sequences corresponding to these measures are feasible in the primal SDP relaxations (10) and (12). To see the compactness observe that the first components (i.e., masses) of the truncated moment vectors y0, y, and ˆy are bounded. This follows by evaluating the equality constraints of (6) and (7) for w(x) = v(x) = 1. Indeed, in discrete time we get (y)0 = (y0)0/(1−α), and in continuous time we get (y)0 = (y0)0/β; in addition, in both cases we have (y0)0+ (ˆy0)0=λ(X)<∞, and therefore the first components are indeed bounded (since they are trivially bounded from below and, in fact, nonnegative, due to the constraints on moment matrices). Boundedness of the even components of each truncated moment vector then follows from the structure of the localizing matrices corresponding to the functions from Assumption 1. Boundedness of the entire truncated moment vectors then follows since the even moments appear on the diagonal of the positive semidefinite moment matrices.
The following result shows the convergence of the optimal values of the finite-dimensional relaxations/approximations to the optimal values of the infinite-dimensional LPs.
Theorem 5. The sequence of infima of LMI problems (11) and (13)converges
monotonically from above to the supremum of the linear programming problems (8)
and (9), i.e., d∗ ≤d∗k+1 ≤d∗k and limk→∞d∗k =d∗ =p∗. Similarly, the sequence of
maxima of LMI problems (10) and (12) converges monotonically from above to the
maximum of the linear programming problems (6) and (7), i.e., p∗ ≤p∗k+1≤p∗k and
limk→∞p∗k=p∗=d∗.
Proof. The monotonicity of the optimal values of the relaxationsp∗k, respectively,
approximations d∗k, is evident from the structure of the feasible sets of the corre-sponding SDPs. The convergence of the primal relaxationspk to p∗ follows from the compactness of the feasible sets of the primal SDPs (10) and (12) (shown in the proof of Lemma 5) by standard arguments on the convergence of Lasserre’s LMI hierarchy (see, e.g., [31]). The convergence of the optimal value of the dual approximationsd∗k to d∗ then follows from Lemma 5. The equality betweenp∗ and d∗ is the statement of Theorems 3 and 4.
The next theorem shows functional convergence from above to the indicator func-tion of the MCI set.
Theorem 6. Let wk ∈R2k[x] denote thew-component of a solution to the dual
LMI problems (11)and (13), and letw¯k(x) = mini≤kwi(x). Thenwk converges from
above toIXI in the L1norm andw¯k converges from above toIXI in theL1norm and
almost uniformly.
Proof. The convergence in the L1 norm follows immediately from Theorem 5
and from the fact that wk ≥ IXI by Lemmas 3 and 4. The convergence of the running minima follows from the fact that there exists a subsequence of {wk}∞k=0
which converges almost uniformly (by, e.g., [4, Theorems 2.5.2 and 2.5.3]).
Our last theorem shows a setwise convergence of the outer approximations to the MCI set.
Theorem 7. Let(vk, wk)∈R2k[x]×R2k[x]denote an optimal solution to the dual
LMI problem (11) or(13), and let XI k:={x∈Rn : vk(x)≥0}. Then XI ⊂XI k,
lim
k→∞λ(XIk\XI) = 0, and λ(∩ ∞
k=1XI k\XI) = 0.
Proof. From Lemmas 3 and 4 we haveXI k⊃XI andwk ≥IXI; therefore, since
w≥v+ 1 andw≥0 onX, we havewk ≥IXI k≥IXI and {x:wk(x)≥1} ⊃XI k ⊃ X0. From Theorem 6, we havewk →IXI in theL1 norm onX. Consequently,
λ(XI) = X IXIdλ= lim k→∞ X wkdλ≥ lim k→∞ X IXI kdλ = lim k→∞λ(XI k)≥klim→∞λ(∩ k i=1XI i) =λ(∩∞k=1XI k).
But sinceXI ⊂XI k for allk, we must have lim
k→∞λ(XI k) =λ(XI) and λ(∩ ∞
k=1XI k) =λ(XI),
and the theorem follows.
7. Numerical examples. In this section we present numerical examples that illustrate our results. The primal SDP relaxations were modeled using Gloptipoly 3 [20] and the dual SOS problems using Yalmip [33]. The resulting SDP problems were solved using SeDuMi [39] (which, in the case of primal relaxations, also returns the dual solution providing the outer approximations). For numerical computation (especially for higher relaxation orders), the problem data should be scaled such that the constraint sets are (within) unit boxes or unit balls; for ease of reproduction, most of the numerical problems shown are already scaled. On our problem class we observed only marginal sensitivity to the values of the discrete- and continuous-time discount factors αand β and report results withα= 0.9 andβ= 1 for all examples presented.
For a discussion on the scalability of our approach and the performance of alter-native SDP solvers see the conclusion and the acrobot-on-a-cart example below.
7.1. Discrete time.
7.1.1. Double integrator. Consider the discrete-time double integrator: x+1 =x1+ 0.1x2,
x+2 =x2+ 0.05u
with the state constraint set X = [−1,1]2 and input constraint set U = [−0.5,0.5]. The resulting MCI set outer approximations of degrees 8 and 12 are shown in Figure 1; the approximation is fairly tight for modest degrees. The true MCI set was computed using the standard algorithm based on polyhedral projections [9].
7.1.2. Cathala system. Consider the Cathala system borrowed from [28]: x+1 =x1+x2,
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 x x d= 8 d= 12
Fig. 1. Discrete time double integrator; polynomial outer approximations (light gray) to the MCI set (dark gray) for degreesd∈ {8,12}.
The chaotic attractor of this system is contained in the setX = [−1.6,1.6]2. MCI set outer approximations are shown in Figure 2; again, the approximations are relatively tight for small relaxation orders. The true MCI set was (approximately) computed by gridding. −1.5 −1 −0.5 0 0.5 1 1.5 −1.5 −1 −0.5 0 0.5 1 1.5 −1.5 −1 −0.5 0 0.5 1 1.5 −1.5 −1 −0.5 0 0.5 1 1.5 x x d= 6 d= 10
Fig. 2. Cathala system; polynomial outer approximations (light gray) to the MCI set (dark gray) for degreesd∈ {6,10}.
7.1.3. Julia sets. Consider over z ∈ C, or, equivalently, over x ∈ R2, with z:=x1+ix2, the quadratic recurrence
z+=z2+c
with c ∈ C a given complex number and i the imaginary unit. The filled Julia set is the set of all initial conditions of the above recurrence for which the trajectories remain bounded. The shape of the Julia set depends strongly on the parameterc. If c lies inside the Mandelbrot set, then the Julia set is connected; otherwise the set is disconnected. In both cases the boundary of the set has a very complicated (in fact, fractal) structure. Here we shall compute outer approximations of the filled Julia set intersected with the unit ball. To this end we setX={x∈R2 : x ≤1}. Figure 3 shows outer approximations of degree 12 for parameter valuesc=−0.7 +i0.2 (inside
c=−0.7 +i0.2 c=−0.9 +i0.2
Fig. 3. Filled Julia set; polynomial outer approximation of degree 12 (light gray) and (an approximation of ) the “true” set (dark grey) represented as an ensemble of initial conditions ran-domly sampled within the state-constraint set. The dashed line shows the boundary of the unit-ball state-constraint set.
the Mandelbrot set) and c =−0.9 +i0.2 (outside the Mandelbrot set). The “true” filled Julia set was (approximately) obtained by randomly sampling initial conditions within the unit ball and iterating the recurrence for one hundred steps. Taking higher degrees of the approximating polynomials does not result in significant improvement due to our choice of the monomial basis to represent polynomials. An alternative basis (e.g., Chebyshev polynomials; see the related discussions in [21] and [19]) would allow us to further improve the outer estimates and better capture the intricate structure of the filled Julia set’s boundary.
7.1.4. H´enon map. Consider the modified controlled H´enon map x+1 = 0.44−0.1x3−4x22+ 0.25u,
x+2 =x1−4x1x2, x+3 =x2,
adapted from [32] with X = [−1,1]3 and U = [−umax, umax]. We investigate two cases: uncontrolled (i.e.,umax = 0) and controlled (with umax = 1). Figure 4 shows outer approximations to the MCI set of degree eight for both settings and the “true” MCI set in the uncontrolled setting (approximately) obtained by random sampling of initial conditions inside the constraint setX. The outer approximations suggest that, as expected, allowing for control leads to a larger MCI set.
7.2. Continuous time.
7.2.1. Double integrator. Consider the continuous-time double integrator ˙
x1=x2, ˙ x2=u,
with state constraint set X = [−1,1]2 and input constraint set U = [−1,1]. The resulting MCI set outer approximations for degrees 8 and 12 are in Figure 5. The
Fig. 4. Controlled H´enon map; polynomial outer approximation of degree eight in the uncon-trolled setting (darker red, smaller) and in the conuncon-trolled setting (lighter red, larger). The (approxi-mation of ) the “true” set (black) in the uncontrolled setting is represented as an ensemble of initial conditions randomly sampled within the state-constraint set.
approximations are fairly tight even for relatively low relaxation orders. The true MCI set was (approximately) computed as in section 7.1.1 by methods of [9] after dense time discretization.
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 x x d= 8 d= 14
Fig. 5.Continuous-time double integrator; polynomial outer approximations (light gray) to the MCI set (dark gray) for degreesd∈ {8,14}.
7.2.2. Spider-web system. As our second example we take the spider-web system from [1] given by equations
˙
x1=−0.15x71+ 200x61x2−10.5x51x22−807x41x32+ 14x31x42+ 600x21x52−3.5x1x62+ 9x72, ˙
with the constraint set X = [−1,1]2. Here we exploit the fact that the system dy-namics are captured by constraints onvonly, whereaswis merely overapproximating v+ 1, and the fact that outer approximations to the MCI set are given not only by
{x:v(x)≥0}but also by{x:w(x)≥1}. Therefore, if low-complexity outer approx-imations are desired, it is reasonable to choose different degrees of v and w in (13) (high forv and lower for w) and use the set{x:w(x)≥1} as the outer approxima-tion. That way, we expect to obtain relatively tight low-order approximations. This is confirmed by numerical results shown in Figure 6. The degree of v is equal to 16 for both figures, whereas degw = 8 for the left figure and degw = 16 for the right figure. We observe no significant loss in tightness by choosing a smaller degree ofw. The true MCI set was (approximately) computed by gridding.
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 x x
Fig. 6.Spider-web system; polynomial outer approximations (light gray) to the MCI set (dark gray) for degreesdegv= 16anddegw= 8on the left anddegw= 16on the right.
7.2.3. Acrobot on a cart. As our last example we consider the acrobot-on-a-cart system adapted from [23], which is essentially a double pendulum on a cart where the inputs are the force acting on the cart and the torque in the middle joint of the double pendulum. The system is sketched in Figure 7. It is a sixth order system with two control inputs; the dynamic equation is given by
˙ x= ⎡ ⎢ ⎢ ⎣ x4 x5 x6 M(x)−1N(x, u) ⎤ ⎥ ⎥ ⎦∈R6, where M(x) = ⎡
⎣a2cosa1 x2 a2cosa4 x2 a5cos(xa3cos2x−3x3) a3cosx3 a5cos(x2−x3) a6 ⎤ ⎦ and N(x, u) = ⎡ ⎣ u1+a2x 2 5sinx2+a3x62sinx3−δ0x4 −a5x26sin(x2−x3) +δ2x6+a7sinx2−x5(δ1+δ2) u2+a5sin(x2−x3)x25+δ2x5−δ2x6+a8sinx3 ⎤ ⎦.
The statesx1, x2,x3 represent, respectively, the position of the cart (in meters), the angle of the lower rod, and the angle of the upper rod of the double pendulum (both in
radians); the statesx4,x5, andx6are then the corresponding velocities in meters per second for the cart and radians per second for the pendulum rods. The constants are given bya1= 0.85,a2= 0.2063,a3= 0.0688,a4= 0.0917,a5= 0.0344,a6= 0.0229, a7 = 2.0233, a8 = 0.6744, δ0 = 0.3, δ1 = 0.1, and δ2 = 0.1. We are interested in computing the maximum controlled invariant subset of the state constraint set
X = [−1,1]×[−π/3, π/3]×[−π/3, π/3]×[−0.5,0.5]×[−5,5]×[−5,5]. We investigate two cases. First, we consider the situation where only the middle joint is actuated and there is no force on the cart; therefore, we impose the constraint (u1, u2) ∈ U ={0} ×[−1,1]. Second, we consider the situation where we can also exert a force on the cart; in this case we impose (u1, u2) ∈ U = [−1,1]×[−1,1]. Naturally, the MCI set for the second case is larger (or at least the same) as for the first case. This is confirmed6by outer approximations of degree four whose section for x1 = 0,x4= 0,x5= 0 is shown in Figure 8. In order to compute the outer approxi-mations, we took a third order Taylor expansion of the nonpolynomial dynamics even though exact treatment would be possible via a coordinate transformation leading to rational dynamics to which our methods can be readily extended; this extension is, however, not treated in this paper, and therefore we opted for the simpler (and non-exact) approach using Taylor expansion. Before solving the problem we made a linear coordinate transform so that the state constraint set becomes the unit hypercube [−1,1]6.
This example, which is the largest of those considered in this paper, took 110 seconds to solve7 with SeDuMi for d= 4; the corresponding time with the MOSEK SDP solver was 10 seconds. Using MOSEK, we could also solve this example ford= 6 (in 420 seconds), although there the solver converged to a solution with a rather poor accuracy,8and therefore we do not report the results.
x1
u1
u2
x2
x3
Fig. 7.Acrobot on a cart; sketch.
8. Conclusion. We derived an infinite-dimensional convex characterization of the maximum controlled invariant (MCI) set, finite-dimensional approximations of (the dual of) which provide a converging sequence of semialgebraic outer approxima-tions to this set. The outer approximaapproxima-tions are the outcome of a single semidefinite
6There is no a priori guarantee on setwise ordering of the outer approximations; what is
guaran-teed is the ordering of optimal values of the optimization problem (12) or (13).
7All examples were run on an Apple iMac with 3.4 GHz Intel Core i7, 8 GB RAM, and Mac OS
X 10.8.2. The time reported is the pure solver time, not including the Yalmip preprocessing time.
8Note that the MOSEK SDP solver is still being developed, and its accuracy is likely to improve
x3 x2 x6= ˙x3 −π/2 −π/4 0 π/4 π/2 −π/2 −π/4 0 π/4π/2 −1 −0.5 0 0.5 1
Fig. 8. Acrobot on a cart; section of the polynomial outer approximations of degree four for
(x1, x4, x5) = (0,0,0). Only the middle joint actuated (darker, smaller); the middle joint and the cart actuated (lighter, larger). The displayed states x2, x3, and x6 are, respectively, the lower pendulum angle, the upper pendulum angle, and the upper pendulum angular velocity.
program (SDP) with no additional data required other than the problem description. Therefore, the approach is readily applicable using freely available modeling tools such Gloptipoly 3 [20] or YALMIP [33] with no hand-tuning involved.
The cost to pay for this comfort is the relatively unfavorable scalability of the SDPs solved—the number of variables grows asO((n+m)d), wherenandmare the
state and control dimensions and d is the degree of the approximating polynomial. Therefore, in order for this approach to scale to medium dimensions (say, more than m+n = 6), one has to either trade off accuracy by taking small d or go beyond the standard freely available solvers such as SeDuMi or SDPA. One possibility is par-allelization; for instance, the free parallel solver SDPARA [49] allows for scaling to larger dimensions. Alternatively, one can utilize one of the (few) commercial SDP solvers; in particular, the recently released MOSEK SDP solver seems to show far superior performance on our problem class, and therefore this may allow scaling to larger dimensions (see also the discussion following the acrobot-on-a-cart example in section 7.2.3). In addition, first-order (e.g., DSA-BD [37]) or augmented Lagrangian methods (e.g., SDPNAL [53]) should allow scaling much further provided that proper conditioning of the problem data is ensured; positive results in this direction have been reported in [38] and are currently investigated further by the authors. Finally, customized structure-exploiting solutions can be developed; this is a promising direc-tion of future research also investigated by the authors. At this point it should be emphasized that, to the best of the authors’ knowledge, all of the existing approaches providing approximations of similar quality experience similar or worse scalability properties.
Other directions of future research include the extension of the presented ap-proach to inner approximations of MCI sets, to stochastic systems, and to uncertain systems. Partial results on the inner approximations for the related problem of ROA computation already exist [27], albeit in an uncontrolled setting only.
