Optimal control and piecewise parametric programming

(1)

Optimal control and piecewise parametric programming

D. Q. Mayne, S. V. Rakovi´c and E. C. Kerrigan

Abstract— This paper deals with the problem of parametric piecewise quadratic programming (in which the cost is a piecewise quadratic function of both the decision variable and a parameter) and the problem of parametric piecewise affine quadratic programming (in which both the cost and the constraint depend on a piecewise affine function of the decision variable and a parameter). Parametric programming seeks a solution for each value of the parameter, and can therefore be used to obtain explicit solutions of some constrained optimal control problems where the state is the parameter. The tech- nique of reverse transformation for parametric programming, introduced in earlier papers, is extended to remove unnecessary overlapping of polytopes on which the solution is defined. The improved technique is then employed for the determination, using dynamic programming, of explicit control for linear systems with a piecewise quadratic cost and explicit control of piecewise affine systems with quadratic cost.

I. INTRODUCTION

A conventional optimization problem has the formV⁰ = minu{V (u) | u ∈ U} where u is the ‘decision’ variable, V (u) is the cost to be minimized, and U is the constraint set;

letu⁰ denote the solution (the minimizer) to the problem. A parametric programming problem takes the form V⁰(x) = minu{V (x, u) | u ∈ U(x)} and both the value function x 7→ V⁰(x) and the minimizer x 7→ u⁰(x) become functions of the parameter x; the minimizer u⁰(·) may be set-valued (for eachx, u⁰(x) is a set). Optimal control problems often take this form, with x being the state, and u, in open-loop optimal control, being a control sequence; in state feedback optimal control, necessary when uncertainty is present, dynamic programming is employed yielding a sequence of parametric optimization problems in each of which x is the state and u a control action. In certain problems, it is possible to characterize both the value function V⁰(·) and the u⁰(·) as functions of a specific type that can be explicitly computed [1]–[13]. When disturbances are present, it is necessary to compute the solution sequentially using dynamic programming as in [3], [7], [11], [13].

To solve the parametric problem, we extend the reverse transformation procedures proposed in [14] and utilized in [3], [6], [8], [9], [15]. In this procedure, a condition that is potentially satisfied at the solution and which, if known, simplifies the solution is assumed to hold. The simplified solution, under this condition, is then obtained, yielding a

Research supported by the Engineering and Physical Sciences Research Council, UK and the Royal Academy of Engineering, UK.

D. Q. Mayne is with the Department of Electrical and Electronic Engineering, Imperial College London, S. V. Rakovi´c is with the Automatic Control Laboratory, ETH, Z¨urich and E. C Kerrigan is with the Department of Aeronautics and the Department of Electrical and Electronic Engineering, Imperial College London

minimizer and value function (functions of the parameter x). Finally, the region of state space in which the assumed condition holds, is then determined; the minimizer and value function of the simplified problem are the minimizer of the original problem in this region. Assuming a different condition yields a different region; in this way, a piecewise affine or piecewise quadratic solution (covering the whole parameter set) to the original parametric problem may be obtained. In parametric quadratic programming, the condition assumed is the set of active constraints at the solution; if this is known, the original inequality constrained problem is converted into an easily solved equality constrained problem.

In the dynamic programming solution of constrained linear systems with piecewise affine or piecewise quadratic cost [15], [16] the value function at each time is piecewise affine or piecewise quadratic; at each stage the cost to be minimized (the sum of the stage cost and the value function at the successor state) is piecewise affine or piecewise quadratic, and the condition assumed is the index that specifies the polytope in which the optimal state-control pair lies (and in which the optimal cost is affine or quadratic). If this condition is known, the original problem is converted into a parametric linear or quadratic program. In control of piecewise affine systems (where the dynamics are affine in each polytope of a polytopic partition of the state-control space) the condition assumed [8], [9] is the time-sequence {s0, s1, . . . , sN −1} of indices that specifies that the optimal state-control pair at time i lies in the polytope Ps_i; if this time-sequence is known, the nonlinear optimal control problem is converted into a simpler, linear, time-varying optimal control problem.

There is, however, a drawback in the reverse transformation procedure currently employed arising from the fact that the simpler problem sometimes involves artificial constraints that do not appear in the original problem. Consequently, a solution to the simpler problem (corresponding to the assumption that the condition is satisfied) may not be a solution to the original problem giving rise to spurious solutions.

Consequently, the regions in which the solution is affine or quadratic may overlap; the optimal solution in an overlapping region must then be determined by comparing the solutions corresponding to each of the overlapping regions. The main purpose of this paper is to present a modified version of the reverse transformation procedure that prevents unnecessary overlapping of regions and removes it entirely in convex problems. We develop the new procedure in the context of two problems: parametric piecewise quadratic programming in§2, its application to the dynamic programming solution of constrained linear systems with quadratic or piecewise quadratic cost in §3, parametric piecewise affine quadratic

(2)

programming in §4 and its application to optimal control of piecewise affine systems in §5. This paper corrects the solution to the first problem that appears in [16].

II. PARAMETRIC PIECEWISEQP A. Introduction

The general parametric programming problem P(x) is defined by:

V⁰(x) = min

u {V (x, u) | (x, u) ∈ Z} (1) u⁰(x) = arg min

u {V (x, u) | (x, u) ∈ Z} (2) where x ∈ Rⁿ,u ∈ R^m (z = (x, u) ∈ Rⁿ× R^m);u⁰(x) is the solution andV⁰(x) the value of problem P(x). Let X , Proj_XZ , {x | ∃ u such that (x, u) ∈ Z}. The following result is established in [17] and earlier in [18] under the stronger hypothesis that the gradients of the active constraints are linearly independent; a self-contained proof is given in [16]:

Proposition 1: SupposeV : Z → R is a strictly convex, continuous function and that Z is a polytope (compact polyhedron). Then, for all x ∈ X = Proj_XZ, the solution u⁰(x) to P(x) exists and is unique. The value function V⁰(·) is strictly convex and continuous with domain X , and the solutionu⁰(·) is continuous on X .

The parametric piecewise quadratic programming problem has more structure and requires a few definitions:

Definition 1: A set {Zi | i ∈ I}, for some index set I, is called a polytopic partition of the polytopic set Z if Z = ∪i∈IZi and the sets Zi, i ∈ I are polytopes with non-intersecting interiors (relative to Z).

Definition 2: A function V : Z → R is said to be piecewise quadratic on a polytopic partition {Zi | i ∈ I}

of Z if it satisfies (for some Qi, qi, ri,i ∈ I)

V (z) = Vi(z), (1/2)z^′Qiz + q^′iz + ri, (3) for allz ∈ Zi, alli ∈ I.

The parametric piecewise quadratic programming problem P(x) (we use the same symbol P(x) for simplicity) is defined by (1) where, now, V (·) is piecewise quadratic and Z is a polytope. The following assumption is assumed to hold in the sequel:

Assumption 1: The function V : Z → R is continuous, strictly convex, and piecewise quadratic on the polytopic partition{Zi| i ∈ I} of the polytope Z in Rⁿ× R^m.

Under this assumption, the piecewise quadratic programming problem P(x) satisfies the hypotheses of Proposition 1 so that its value function V⁰(·) is strictly convex and continuous and the minimizer u⁰(·) is continuous. Assumption 1 implies that Qi is positive definite for alli ∈ I. For each x, let the set U(x) be defined by

U(x), {u | (x, u) ∈ Z}. (4) ThusU(x) is the set of admissible u at x and P(x) may be expressed in the form V⁰(x) = minu{V (x, u) | u ∈ U(x)}.

Because of the piecewise nature ofV (·), we require:

Definition 3: A polytopeZiin a polytopic partition{Zi| i ∈ I} of a polytope Z is said to be active at z ∈ Z if z = (x, u) ∈ Zi. The set of polytopes active atz ∈ Z is

S(z), {i ∈ I | z ∈ Zi}. (5) A polytope Zi in a polytopic partition {Zi | i ∈ I} of a polytope Z is said to be active for Problem P(x) if (x, u⁰(x)) ∈ Zi. The set of active polytopes for P(x) is S⁰(x) defined by

S⁰(x), S(x, u⁰(x)). (6) Thus S⁰(x) = {i ∈ I | (x, u⁰(x)) ∈ Zi}. To proceed, and to relate our version of reverse transformation to earlier versions, we define, for each i ∈ I, the subproblem Pi(x) employed in these versions and defined by

Vi⁰(x) = min

u {V (x, u) | (x, u) ∈ Zi} (7) u⁰i(x) = arg min

u {V (x, u) | (x, u) ∈ Zi}. (8) As before, the set of feasible u for Pi(x) is Ui(x) , {u | (x, u) ∈ Zi}.

The question arises: how are the solutions to Pi(x), i ∈ S⁰(x) related to the solution of the original problem P(x)?

This is answered by [16]:

Proposition 2: Suppose x ∈ X is given. The following two statements are equivalent:

(i)u is optimal for the original problem P(x) (u = u⁰(x)).

(ii) u is optimal for problem Pi(x) for all i ∈ S⁰(x) (u = u⁰_i(x) for all i ∈ S⁰(x)).

A proof of this result appears in [16] but is included here since it is simple and motivates the algorithm given later.

Proof: (i) Supposeu is optimal for P(x); then, (x, u) ∈ Zi

for alli ∈ S⁰(x). If u is not optimal for Pi(x) for some i ∈ S⁰(x), there exists a v such that (x, v) ∈ Zi andV (x, v) = Vi(x, v) < Vi(x, u) = V (x, u) = V⁰(x), a contradiction.

(ii) Suppose u is optimal for Pi(x) for all i ∈ S⁰(x). Then V (x, u) = Vi(x, u) ≤ V (x, u⁰(x)) = V⁰(x) for all i ∈ S⁰(x). But V⁰(x) ≤ V (x, u) (by the optimality of u⁰(x)), so thatV (x, u) = V⁰(x) which establishes the optimality of u for P(x).

For eachi ∈ I, Zi may be defined by

Zi, {Mⁱu ≤ Nⁱx + pⁱ}. (9) LetM_jⁱ,N_jⁱ andpⁱ_j denote, respectively thejth row of Mⁱ, Nⁱ andpⁱ. Let

Ii(x, u), {j | Mjⁱu = N_jⁱx + pⁱ_j} (10) denote the active constraint set for Pi(x) at (x, u) ∈ Zi. Let Fi(x, u) denote the set of feasible directions at u ∈ Ui(x), {u | (x, u) ∈ Zi}:

Fi(x, u), {h ∈ R^m| M_jⁱh ≤ 0 ∀j ∈ Ii(x, u)}, (11)

(3)

and letP Ci(x, u) denote the polar cone of Fi(x, u) at 0:

P Ci(x, u), {v ∈ R^m| v^′h ≤ 0 ∀h ∈ Fi(x, u)}

=





 X

j∈Ii(x,u)

(M_jⁱ)^′λj

λ^j ≥ 0 ∀j ∈ Ii(x, u)







. (12)

Using a well-known condition of optimality employed in [8], [19] in this context we have:

Proposition 3: u is optimal for problem Pi(x) if and only ifu ∈ Ui(x) and −∇uVi(x, u) ∈ P Ci(x, u).

B. Improved reverse transformation algorithm

Proposition 2 shows that solving, as in the original version of the reverse transformation procedure, Problem Pi(x) if polytope Zi is active, does not necessarily yield a solution of the original problem; it does so if Zi is the sole active polytope.

For anyx ∈ X , u⁰(x) (the solution of P(x)) satisfies the equality constraint

M_jⁱu = N_jⁱx + pⁱ_j, ∀j ∈ I_i⁰(x), ∀i ∈ S⁰(x) (13) where M_jⁱ denotes the jth row of Mⁱ, etc, and I_i⁰(x) , Ii(x, u⁰(x)) indexes those constraints in (9) that are active at(x, u⁰(x)). For simplicity, we rewrite (13) as

Exu = Fxx + gx. (14) Henceu⁰(x) is the unique solution of

V⁰(x) = min

u {V (x, u) | Exu = Fxx + gx}.

We define, for allx, ¯x ∈ X , the (easy) equality constrained problem Px(¯x) by

Vx⁰(¯x) = min

u {V (¯x, u) | Exu = Fxx + g¯ x} (15) u⁰x(¯x) = arg min

u {V (¯x, u) | Exu = Fxx + g¯ x}. (16) where V (¯x, u) = Vi(¯x, u), i ∈ S⁰(x) and is, therefore, quadratic, for all(¯x, u) satisfying (14). The solution to this problem is

V_x⁰(¯x) = (1/2)¯x^′Qxx + s¯ xx + r¯ x (17) u⁰_x(¯x) = Kxx + k¯ x. (18) where Qx, sx, rx, Kx, kx are easily computed. The minimizer u⁰_x(¯x) to Px(¯x) satisfies (14), the same equality constraints as those satisfied by u⁰(x). We know that u⁰x(·) is optimal for Px(¯x) and is also optimal for P(x); the next result shows that u⁰x(·) is also optimal for P(¯x) for all ¯x in some regionRx⊂ Rⁿ.

Proposition 4: Letx be an arbitrary point in X . Then: (i) The solution to P(x) satisfies u⁰(¯x) = u⁰_x(¯x) and V⁰(¯x) = V_x⁰(¯x) for all ¯x ∈ Rx defined by:

Rx,

¯ x

u⁰x(¯x) ∈ Ui(¯x) ∀i ∈ S⁰(x)

−∇uVi(¯x, u⁰x(¯x)) ∈ P Ci⁰(x) ∀i ∈ S⁰(x)

(19) where P C_i⁰(x), P Ci(x, u⁰(x)), (ii) Rx is a polytope, and (iii) x ∈ Rx.

This proposition is an elementary consequence of Propo- sition 3 and corrects an earlier version in [16]. Since there are a finite number of polytopes Zi and a finite number of constraints defining each polytope, there exists a finite set of points{xj, j ∈ J } in X such that X = ∪j∈JRxj. Hence, we have

Theorem 1: There exists a finite set of points{xj, j ∈ J } inX such that R = {Rxj | j ∈ J } is a polytopic partition of X . The value function V⁰(·) is piecewise quadratic and the minimiser u⁰(·) is piecewise affine in R, being equal, respectively, toV_x⁰_j(·) and u⁰_x_j(·) in each region Rx_j,j ∈ J .

Propositions 3 and 4 motivate:

Improved Reverse Transformation Algorithm 1. Initialize: SetR = ∅.

2. Update: Select x ∈ X \ R, solve P(x). Determine the affine minimizer u⁰_x(·), the quadratic value function V_x⁰(·) and the polytopeRx. Set V⁰(·) = V_x⁰(·) and u⁰(·) = u⁰_x(·) inRx. SetR = Rx∪ R.

3. Iterate: WhileR 6= X , iterate.

The algorithm generates a sequenceRxj,j ∈ J of polytopes that form a polytopic partition ofX and yields the solution (both the value function and the minimizer) to the parametric piecewise quadratic problem P(x) for all x ∈ X .

We can compare the original and improved version of reverse transformation by their application to the example shown in Figure 1. HereZ is the rectangle X × U.

Z1

Z2

x u

u⁰₁(·)

u⁰₂(·)

U X

(a) Solutions of P1(x) and P2(x)

Z1

Z2

x u

u⁰(·) U

X

(b) Solution of P(x)

Fig. 1. Improved reverse transformation algorithm

The polytopeZ is partitioned into two polytopes Z1andZ2

in each of whichV (·) is quadratic (in z = (x, u)).

The original version yields u⁰₁(·) and u⁰₂(·) shown in Figure 1(a); bothu⁰₁(x) and u⁰₂(x) exists at each x ∈ X and further investigation is required to choose the true minimizer u⁰(x). The improved algorithm, in contrast, yields the unique minimizeru⁰(·) shown in Figure 1(b).

III. DYNAMIC PROGRAMMING SOLUTION OF THE

CONSTRAINEDLQRPROBLEM

This problem was considered in [15] using an earlier version of reverse transformation. Consider the problem of controlling the linear system

x⁺= Ax + Bu (20)

(4)

where x and u are the current state and control, and x⁺ is the successor state. Let u, {u(0), u(1), . . . , u(N −1)}. We pose an optimal control problem P(x) defined by

VN⁰(x) = min

u VN(x, u) (21)

u⁰_N(x) = arg min

u

VN(x, u) (22)

subject to the constraints

x(i) ∈ X, u(i) ∈ U, i = 0, . . . , N − 1, x(N ) ∈ Xf, (23) wherex(i), φ(i; x, u) is the solution at time i of (20) with initial statex at time 0 and control sequence u. The sets X, U and Xf are assumed to be polytopic. The cost function VN(·) is defined by

VN(x, u),

N −1

X

i=0

ℓ(x(i), u(i)) + Vf(x(N )) (24) where ℓ(·) and Vf(·) are continuous strictly convex piecewise quadratic functions defined, respectively, on polytopic partitions of X× U and Xf. The complexity of the problem, especially when ℓ(·) is piecewise quadratic, suggests that dynamic programming is more efficient whenN is large. Let f (x, u), Ax + Bu. The constrained dynamic programming recursion is

V_i+1⁰ (x) = min

u {ℓ(x, u) + V_i⁰(f (x, u)) | u ∈ U, f (x, u) ∈ Xi}, (25) Xi+1= {x ∈ X | ∃ u ∈ U such that

f (x, u) ∈ Xi} (26) with boundary conditions

V₀⁰(·) = Vf(·), X0= Xf (27) It is easily seen that each stage of the dynamic pro- gramming recursion is a parametric piecewise quadratic programming problem (even if ℓ(·) is quadratic) to which the algorithm described above may be usefully applied with V⁰(·) = V_i+1⁰ (·), V (x, u) = ℓ(x, u) + V_i⁰(Ax + Bu) (which is piecewise quadratic in (x, u) since ℓ(·) and V_i⁰(·) are piecewise quadratic) and Z = {(x, u) | x ∈ X, u ∈ U, Ax + Bu ∈ Xi} (which is polytopic since U, X and Xi

are polytopic).

IV. PARAMETRIC PIECEWISE AFFINEQP

The parametric piecewise affine quadratic program P(x) is defined by

V⁰(x) = min

u {V (x, u) | (x, u) ∈ Z} (28) where

V (x, u), ℓ(x, u) + Vf(f (x, u)),

Z, {(x, u) ∈ Z0| f (x, u) ∈ Xf}. (29) It is assumed that the cost functionsℓ(·) and Vf(·) are strictly convex and quadratic, that Z0 and Xf are polytopic and that f (·) is continuous and piecewise affine in a polytopic

partitionP = {Pi| i ∈ I} of Z0 so that, for eachi ∈ I f (x, u) = fi(x, u), Aix + Biu + ci ∀(x, u) ∈ Pi (30) The minimizer for P(x) (see (28)) is u⁰(x) and may be set-valued. The domain of V⁰(·) is X , {x |

∃ u such that (x, u) ∈ Z}. The complicating factor (com- pared with the piecewise quadratic problem) is f (·) which renders the problem non-convex. For eachi ∈ I, define the setZi ⊆ Pi by

Zi, {(x, u) ∈ Pi| fi(x, u) ∈ Xf} (31) Sincefi(·) is affine, each Zi is a polytope andZ = ∪i∈IZi

is a polygon (union of a finite set of polyhedra). Also, for eachi ∈ I, define the cost Vi: Zi→ R by

Vi(x, u), ℓ(x, u) + V^f(fi(x, u)) ∀(x, u) ∈ Zi. (32) Sincefi(·) is affine, Vi(·) is quadratic and is strictly convex.

Thus, for eachi ∈ I, problem Pi(x), defined by V_i⁰(x) = min

u {Vi(x, u) | (x, u) ∈ Zi} (33) is a quadratic program withVi(·) strictly convex so that, for each x ∈ Proj_XZi, Pi(x) has a unique global minimizer u⁰_i(x). Let

Ui(x), {u | (x, u) ∈ Zi}. (34) The domain ofV_i⁰(·) and u⁰_i(·) is Xi= {x | Ui(x) 6= ∅}.

Necessary and sufficient conditions for the optimality of u for Pi(x) are u ∈ Ui(x) and −∇uVi(x, u) ∈ P Ci(x, u) where, now,P Ci(x, u) is the polar cone of the set of feasible directions forZi at0.

SinceV (x, u), ℓ(x, u)+Vf(f (x, u)), we have V (x, u) = Vi(x, u) for all (x, u) ∈ Zi. Hence V (·) is piecewise quadratic and continuous but not continuously differentiable;

V (·) is not necessarily convex so that, for each x, the global minimizer is not necessarily unique, and V (·) may have several local minima though each local minimizer of P(x) is a global minimizer of Pi(x) for some i ∈ I. Also let S(x, u) be defined as in (5). Because V (·) is not necessarily convex, u⁰(x) may be set-valued and a local minimum is not necessarily global; however u⁰(x) ⊆ {u⁰_i(x) | i ∈ I} and has finite cardinality. Hence Proposition 2 needs modification. We have, instead

Proposition 5: Let x ∈ X be given. If u ∈ u⁰(x) is optimal for P(x), then u is optimal for Pi(x) for all i ∈ S(x, u).

The proof of this result is similar to the proof of part (i) of Proposition 2. Similarly to (13), anyx ∈ X , any u ∈ u⁰(x) satisfies

Mjⁱu = Njⁱx + pⁱj, ∀j ∈ Ii(x, u), ∀i ∈ S(x, u) which may be written as

E(x,u)u = F(x,u)x + g(x,u).

For allx ∈ X , all u ∈ u⁰(x), we define as before, for all

(5)

¯

x ∈ X , the equality constrained problem P(x,u)(¯x) by V_(x,u)⁰ (¯x) = min

v {V (¯x, v) | E(x,u)v = F(x,u)x + g¯ (x,u)} (35) u⁰_(x,u)(¯x) = arg min

v {V (¯x, v) | E(x,u)v = F(x,u)¯x + g(x,u)}.

(36) As before, V_(x,u)⁰ (·) is quadratic and u⁰_(x,u)(·) is affine and unique (as in (15) and (16)) and are easily computed.

For eachx ∈ X , each u ∈ u⁰(x), let the region R(x,u)be defined as before:

R(x,u), {¯x | u⁰(x,u)(¯x) ∈ Ui(¯x), −∇uVi(¯x, u⁰_(x,u)(¯x))

∈ P Ci(x, u) ∀i ∈ S(x, u)} (37) whereP Ci(x, u) is defined as in (9)–(12). As before, R(x,u)

is a polytope and x ∈ R(x,u). For each x ∈ R¯ (x,u), each

¯

u ∈ u⁰(¯x), (¯x, ¯u) ∈ Zi andu minimizes V¯ i(¯x, u) in Ui(¯x) for alli ∈ S(¯x, ¯u). Because u⁰_(x,u)(·) is now a local, rather than a global, minimizer, it is possible for the regionsR_(x,u) to overlap. This is illustrated in Figure 2 where P1 and P2 are, respectively, the polytopes above and below the diagonal and Z0 = P1 ∪ P2. The polytopes Z1 and Z2, which are, respectively, subsets of P1 andP2, are disjoint.

The sets R(x₁,u₁) where u1 ∈ u⁰(x1) and R(x₂,u₂) where u2 ∈ u⁰(x2) overlap. Consider a point ¯x lying in, say, an overlap region R = ∩j∈KR(xj,uj) for some index set K.

Then each u⁰_(x_j_,u_j₎(¯x), j ∈ K, is a local minimizer and at least one is a global minimizer for P(¯x). Hence, if ¯x ∈ R, u⁰_(x

j∗,u_j∗)(¯x) ∈ u⁰(¯x) for each j^∗∈ arg minj{V (¯x, u⁰_j(¯x)) | j ∈ K} and we can no longer claim that that the value functionV⁰(·) for P(x) is piecewise quadratic and that the minimizer u⁰(·) is piecewise affine on a polytopic partition ofZ since the boundaries between the sub-regions of R may be parabolic. Summarizing:

x u

P1

P2

x1 x2

R_(x₁,u₁)

R(x₂,u₂)

R

Z0

Z1

Z2

u1

u2

Fig. 2. Overlapping regions: R= R_(x₁_,u₁₎∩ R_(x

2,u₂)

Proposition 6: The set of states for which P(x) has a solution is a polygon (union of a finite set of polyhedra).

There exists a finite set of points{(xj, uj) | j ∈ J } such that X = ∪j∈JR(xj,u_j). Ifx ∈ ∩¯ j∈KR(xj,u_j) for someK ⊂ J , then V⁰(¯x) = V (¯x, u⁰j^∗(¯x)) and u⁰j^∗(¯x) ∈ u⁰(¯x) for any j^∗∈ arg minj{V (¯x, u⁰_j(¯x)) | j ∈ K}.

Despite the non-convexity of V (·) and the non-polytopic

nature ofZ we can employ a modification of the algorithm, described in §2, to obtain a set of polytopes of the form R(x,u), each lying in X = Proj_XZ. Because of non- convexity, regions R(xj,u_j), j ∈ J , may overlap. Also, because points in R merely satisfy, in general, a necessary condition of optimality, there may exist points x in Rk =

∪j∈J_kR(x_j,u_j) (at the kth iteration of the algorithm) such that V⁰(x) < minj{V_(x⁰

j,u_j)(x) | j ∈ Jk}. Hence the successor point x in Step 2 of the algorithm should not¯ necessarily be sought in X \ Rk otherwise these points may be overlooked. However, the number of overlaps is considerably reduced with the improved algorithm because false minimizers lying on internal boundaries separating one polytope from another are avoided. See Figure 3 where u7→ V (x, u) is plotted for a given value x of the parameter.

The original version identifies ua, uband uc(twice) as local minimizers whereas the improved version identifies ua and u_c. The results of this section still hold if ℓ(·) and Vf(·)

u_a u_b u_c u

V (x, u)

Fig. 3. Non-convex problem

are piecewise quadratic provided P is appropriately sub- partitioned.

V. OPTIMAL CONTROL OF A PIECEWISE AFFINE SYSTEM WITH QUADRATIC COSTS

Consider the problem of controlling the piecewise affine system described by

x⁺= f (x, u) (38)

subject to the constraints

(x(i), u(i)) ∈ Z0, i = 0, 1, . . . , N − 1, x(N ) ∈ Xf (39) where x(i), φ(i; x, u), Z0 andXf are polytopic andf (·) is continuous and piecewise affine satisfying:

f (x, u) = Ajx + Bju + cj ∀(x, u) ∈ Pj (40) where P = {Pi | i ∈ I} is a polytopic partition of Z. The cost is, as before,

V (x, u) =

N −1

X

i=0

ℓ(x(i), u(i)) + Vf(x(N )) (41) where N is the horizon, u denotes the control sequence {u(0), u(1), . . . , u(N − 1)} and x(i) = φ(i; x, u); ℓ(·) and Vf(·) are assumed, for simplicity, to be quadratic. For each s, {s(0), s(1), . . . , s(N − 1)} ∈ S , I^N we define

Zs, {(x, u) | (φ(i; x, u), u(i)) ∈ Ps(i), i = 0, . . . , N −1;

φ(N ; x, u) ∈ Xf}. (42)

(6)

and define Z to be the polygon ∪s∈SZs. We now observe that, if (x, u) ∈ Z^s, then the (nonlinear) piecewise affine system behaves like the time-varying system

x(t + 1) = As(t)x(t) + Bs(t)u(t) + cs(t) (43) so thatV (·) is quadratic (V (x, u) = V^s(x, u) where V^s(·) is quadratic) for all(x, u) ∈ Z^s. Thus, for each s∈ S, problem P_s, defined by

Vs⁰(x) = min

u

{Vs(x, u) | (x, u) ∈ Z^s} (44) is a quadratic program. It can now be seen that the piecewise affine optimal control problem is identical to the problem discussed in§4 with u replacing u and Zs, s ∈ S replacing Zi, i ∈ I. Hence the results obtained in §4 apply with obvious modifications to the piecewise affine optimal control problem considered here.

VI. CONCLUSION

A procedure for parametric piecewise quadratic program- ing that is an improvement on earlier versions of reverse transformation is described. It removes false candidates for the minimizer that arise in the earlier versions because minimizers for the simplified problems lying on the boundary between polytopes do not necessarily satisfy conditions of optimality for the original problem. The resultant procedure removes overlapping of regions entirely in convex problems (such as the dynamic programing problem for constrained linear quadratic control discussed in §3) and reduces the number of overlaps in non-convex problems (such as the optimal control problem for piecewise affine systems discussed in§5).

REFERENCES

[1] A. Bemporad, F. Borrelli, and M. Morari, “Piecewise linear optimal controllers for hybrid systems,” in Proceedings of the American Control Conference, Chicago, 2000, pp. 1190–1194.

[2] ——, “Optimal controllers for hybrid systems: stability and piecewise linear explicit form,” in Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, December 2000.

[3] D. Q. Mayne, “Control of constrained dynamic systems,” European Journal of Control, vol. 7, pp. 87–99, 2001.

[4] D. Ramirez and E. Camacho, “On the piecewise linear nature of min-max model predictive control with bounded uncertainties,” in Proceedings of the 40th IEEE 2001 Conference on Decision and Control, Orlando, Florida, USA, 2001, pp. 4845–4850.

[5] A. Bemporad, M. Morari, V. Dua, and E. Pistikopoulos, “The ex- plicit linear quadratic regulator for constrained systems,” Automatica, vol. 38, no. 1, pp. 3–20, 2002.

[6] F. Borrelli, “Discrete time constrained optimal control,” Ph.D. disser- tation, Swiss Federal Instritute of Technology, Zurich, 2002.

[7] E. C. Kerrigan and D. Q. Mayne, “Optimal control of constrained piecewise affine systems with bounded disturbances,” in Proceedings of the 41st IEEE 2002 Conference on Decision and Control, vol. 2, Las Vegas, USA, 2002, pp. 1552 – 1557.

[8] D. Q. Mayne and S. Rakovi´c, “Optimal control of constrained piecewise affine discrete-time systems using reverse transformation,” in Proceedings of the IEEE 2002 Conference on Decision and Control, vol. 2, Las Vegas, USA, 2002, pp. 1546 – 1551 vol.2.

[9] ——, “Optimal control of constrained piecewise affine discrete-time systems,” Journal of Computational Optimization and Applications, vol. 25, no. 1-3, pp. 167–191, 2003.

[10] M. M. Seron, G. C. Goodwin, and J. A. D. Don´a, “Characterisation of receding horizon control for constrained linear systems,” Asian Journal of Control, vol. 5, no. 2, pp. 271–286, June 2003.

[11] A. Bemporad, F. Borrelli, and M. Morari, “Min-max control of constrained uncertain discrete-time linear systems,” IEEE Transactions on Automatic Control, vol. 48, no. 9, pp. 1600–1606, September 2003.

[12] E. C. Kerrigan and J. M. Maciejowski, “Feedback min-max model predictive control using a single linear program: Robust stability and the explicit solution,” International Journal of Robust and Nonlinear Control, vol. 14, no. 4, pp. 395–413, March 2004.

[13] M. Diehl and J. Bjornberg, “Robust dynamic programming for min- max model predictive control of constrained uncertain systems,” IEEE Trans. Automatic Control, vol. 49, no. 12, pp. 2253–2257, December 2004.

[14] D. Q. Mayne, “Control of constrained dynamic systems,” Imperial College London, Tech. Rep. EEE/C&P/DQM/9/2001, 2001, keynote address, European Control Conference, Oporto, 4–7 September, 2001.

[15] D. M. de la Pe˜na, T. Alamo, A. Bemporad, and E. F. Camacho, “A dynamic programming approach for determining the explicit solution of MPC controllers,” in Proceedings 43rd IEEE Conference on Decision and Control, Paradise Island, Bahamas, December 2004.

[16] D. Q. Mayne, S. V. Rakovi´c, R. B. Vinter, and E. C. Kerrigan,

“Characterization of the solution to a constrained H∞optimal control problem,” Automatica, vol. 42, pp. 371–382, 2006.

[17] B. Bank, J. Guddat, D. Klatte, B. Kummer, and K. Tanner, Non-linear parametric optimization. Basel, Boston, Stuttgart: Birkh¨auser Verlag, 1983.

[18] G. B. Dantzig, J. Folkman, and N. Z. Shapiro, “On the continuity of the minimum set of a continuous function,” Journal of Mathematical Analysis and Applications, vol. 17, no. 3, pp. 519–548, 1967.

[19] D. Q. Mayne and S. Rakovi´c, “Model predictive control of constrained piecewise affine discrete-time systems,” International Journal of Ro- bust and Nonlinear Control, vol. 13, no. 3-4, pp. 261–279, 2003.