www.elsevier.com/locate/automatica
Technical Communique
On smooth optimal control determination
夡
Ilya Ioslovich
∗, Per-Olof Gutman
Faculty of Civil and Environmental Engineering, Lowdermilk Division of Agricultural Engineering, Technion, Haifa 32000, Israel Received 15 January 2003; received in revised form 28 May 2004; accepted 15 July 2004
Abstract
When using the Pontryagin Maximum Principle in optimal control problems the most difficult part of the numerical solution is associated with the non-linear operation of the maximization of the Hamiltonian over the control variables. For a class of problems, the optimal control vector is a vector function with continuous time derivatives. A method is presented to find this smooth control without the maximization of Hamiltonian. Three illustrative examples are considered.
䉷2004 Elsevier Ltd. All rights reserved.
Keywords: Optimal control; Rigid body rotation; Crop density
1. Description of the method
Let us consider the classical optimal control problem (OCP) (Pontryagin, Boltyansky, Gamkrelidze, & Mis-chenko, 1962;Lee & Markus, 1967;Athans & Falb, 1966;
Pinch, 1993), in the form T
0
f0(x, u)dt →min, (1)
dx
dt =f (x, u), (2)
x(0)=x0, x(T )=xT, (3)
where the control variables u(t)∈ Rm, the state variables
x(t) ∈ Rn, and f (x, u) ∈ Rn are column vectors, with
mn. The right-hand-side functions of the state equation (2), f (x, u) and the performance index function f0(x, u)
are smooth over all arguments. It is assumed that there are no other constraints except (3). The column vector
p(t) ∈ Rn is the vector of costate variables. According to
夡This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Assistant Editor Rick Middleton under the direction of Editor P. Van den Hof.
∗Corresponding author. Tel.: +972-4-8294254; fax: +972-4-8295696. E-mail address:[email protected](I. Ioslovich).
0005-1098/$ - see front matter䉷2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.automatica.2004.07.011
Pontryagin’s Maximum Principle (PMP) (Pontryagin et al., 1962) the Hamiltonian H, is defined as
H=pTf (x, u)−p
0f0(x, u). (4)
Let us consider the “normal” case when p0>0. Without
loss of generality we setp0=1, leading to
H=pTf (x, u)−f
0(x, u). (5)
According to PMP the following system of differential equa-tions for the co-state variables p holds:
dp dt = −
jH
jx
T = −jf
jx
T
p+jf0
jx
T
. (6)
If there exists an optimal solution(x∗, u∗), then, according to PMP, there exists a costate vectorp∗such that the following conditions are satisfied:
(a) H (x∗, u∗, p∗)H (x∗, u, p∗), meaning that H has maximum over the control u,
(b) the variablesx∗, p∗satisfy Eqs. (2), (6), (c) the end conditions (3) must be hold.
time, u is determined from condition (a). There are as many unknown valuesp(0)as given end conditionsx(T )=xT. The maximization problem in (a) is often computationally costly. Since no other constraint than (3) is present, it follows from (a) that the control vector u∗ has to satisfy the necessary conditions
jH
ju =0. (7)
Often these equations are non-linear inu∗. In the following, we shall omit the symbol * for simplicity. From (5) and (7) one gets the system of equations
pTjf
ju −
jf0
ju =0, (8)
wherejf0/juis a row vector, and the dimension of the
Ja-cobianjf/juisn×m.The Eq. (8) is linear in p. As men-tioned above, according to the usual procedure for solving an OCP with PMP, the non-linear system of m equations (8) must be solved in order to find the control vector u as a function of the costate vector p and the state vector x. In contrast, by linear operations on (8) it is very easy to com-pute m elements of the costate vector p as a vector function of the state vector x, the control vector u, and the n−m remaining costates. Let us assume that the rank of the Ja-cobian matrix jf/ju is equal to m. This means that there exists a non-singularm×msub-matrix. We shall re-index the state variables in such a manner that the m variables cor-responding to this sub-matrix have indices 1 to m, and we shall denote the sub-vectors of the corresponding right-hand functions in (2) asfa, the costates aspa, the sub-vector of the remaining right-hand functions asfb, and the remaining costates aspb. Consequently,
x= [xa;xb]; p= [pa;pb];
f (x, u)= [fa(x, u);fb(x, u)].
Eq. (8) can be rewritten as
paTjfa
ju +pb
Tjfb
ju −
jf0
ju =0. (9)
From here follows that
pa=
jfa ju
T−1 −jfb
ju
T
pb+jf0
ju
T .
=A(x, pb, u),
(10) where for convenience we have defined the vector function
A(x, pb, u). Taking the derivative of the costate pa from
(10) with respect to time t, one gets dpa
dt =
jA
jxf (x, u)+
jA
ju
du dt +
jA
jpb
dpb
dt . (11) The differential equations for the costates (6) can be rewrit-ten as
dpa dt = −
jH
jxa
T
= −jfa
jxa
T
pa−jfb
jxa
T
pb+jf0
jxa
T
, (12)
dpb dt = −
jH
jxb
T
= −jfa
jxb
T
pa−jfb
jxb
T
pb+jf0
jxb
T
. (13)
By substituting the expressions for pa from (10) into the right-hand sides of (12), (13), one gets
dpa dt = −
jfa jxa
T
A−jfb
jxa
T
pb+jf0
jxa
T
, (14)
dpb dt = −
jfa jxb
T
A−jfb
jxb
T
pb+jf0
jxb
T
. (15)
For convenience we shall denote the right-hand side of Eq. (15) asS(x, pb, u), namely,
dpb dt = −
jfa jxb
T
A−jfb
jxb
T
pb+jf0
jxb
T .
=S(x, pb, u).
(16) Exploiting the fact that the right-hand side of (14) is equal to the right-hand side of (11), one gets, with a simple matrix transformation, the equations for the time derivatives du/dt as functions of x,pb, and u only, as follows
B .=jA
ju.
We shall assume that B is non-singular. Thus,
jA
jxf (x, u)+
jA
jpb
dpb dt +B
du dt
= −jfa
jxa
T
A−jfb
jxa
T
pb+jf0
jxa
T
,
and hence, taking into account (16), one gets
du dt =B
−1
−jfa
jxa
T
A−jfb
jxa
T
pb+jf0
jxa
T
− jA
jxf (x, u)
jA
jpbS(x, pb, u)
.
=F (x, pb, u), (17)
where the notationF (x, pb, u)is defined. Thus, the problem is reduced to integrating the system of equations
du
dt =F (x, p
b, u),
dx
dt =f (x, u), dpb
dt =S(x, p
b, u), (18)
with the initial values x(0)=x0, and instead of the pair
of initial values pa(0), pb(0), the appropriate pair of ini-tial valuesu(0), pb(0)are to be found. The end conditions
have the differential equations for u. Note that no maximiza-tion of the Hamiltonian, nor any non-linear operamaximiza-tions are needed for the determination ofu(t). Instead, just a number of linear matrix operations have to be performed, which is a much easier task. The following theorem can be formulated based on the notations and derivations given above.
Theorem. If the OCP problem (1)–(3),mn,has the op-timal solutionx∗, u∗such thatu∗is smooth, and if also the problem is normal in the sense that the costatep0∗ can be set to 1, the Jacobianjfa/juis non-singular, the Jacobian B is non-singular, then the optimal states,x∗, costates,p∗b,
and controlu∗satisfy the Eqs. (18).
Note. In the case when n=m the set of variables and costatesxb, pbis empty and Eqs. (18) are changed accord-ingly. In this case the last set of these equations, namely those related topb,vanishes.
2. Rigid body rotation
As the first example, let us consider the following axisym-metric rigid body rotation problem, which is a modification of the problem considered inAthans and Falb (1966)
dx
dt =ay+u1, dy
dt = −ax+u2. (19) The variables x and y are the components of the angular velocity that need to be stopped, i.e., the final conditions are
x(T )=0, y(T )=0, (20)
and final time T is given. The performance index to be min-imized is chosen as
J=1
4
T
0
(u2 1+u
2 2)
2dt →min.
Here, according to (10), the vector A has the form
p=A= u
1(u21+u22)
u2(u21+u22)
. (21)
The costate equations are dpx
dt =apy, dpy
dt = −apx. (22)
One may notice that the Hamiltonian is strictly concave. HerejA/jx=0. The matrix B is symmetric and has the form
B=
3u21+u22 2u1u2
2u1u2 u21+3u22
(23)
with
B−1= 1
(3[u21+u22]2)
u2
1+3u22 −2u1u2
−2u1u2 3u21+u22 . (24)
Extracting the time derivatives foru1, u2one gets after some
simplifications
F = du1
dt du2
dt
=
au2 −au1
, (25)
implying du1
dt =au2, du2
dt = −au1. (26)
From (26) it is easily seen that
u2
1+u22=C2=const.
Hence the control vector (u1, u2)T has constant length
C >0, and it rotates around the origin of the coordinate system in the u-plane with the constant angular velocity a. Introducing the variablesr,by the coordinate transforma-tion
x=r sin
y=r cos (27)
one realizes that the system of Eqs. (19), (26) has the solution
u1= −C x
x2+y2,
u2= −C y
x2+y2. (28)
Thus, the initial value of the vectoru= [u1;u2]is collinear and has opposite direction to the initial vector of the state coordinates [x;y]. To check the admissibility of (28), one may substitute (27) into (19) and find the equations for the time derivatives of r,. To find the derivative of r, the re-sulting equations must be multiplied by sin,cos, respec-tively, and then summed. To find the derivative ofthe same equations must be multiplied by cosand−sin, respec-tively, and then summed. Substituting (28) into the obtained equations one gets
dr
dt = −C, d
dt =a. (29)
Thus one can see that both vectors(u1, u2)Tand(x, y)T
ro-tate with the same angular velocity a and their collinearity is maintained in steady state. Let us denote
x(0)2+y(0)2=
3. Optimal spacing for greenhouse lettuce growth
Let us consider the problem of the optimal variable spac-ing policy for greenhouse lettuce growth (seeIoslovich & Gutman, 1999, 2000; Seginer & Ioslovich, 1999). The dynamical model of the lettuce growth in constant climate conditions has the form
dv
dt =aG(W), (30)
wherevis the dry mass of a single plant (kg/plant), a is the spacing(m2/plant), G is the net photosynthesis(kg/m2), W is the plant density(kg/m2). The net photosynthesisG(W) is a strictly concave function. The spacing is variable in time and must be adjusted in an optimal way during the growth process to minimize the cost of the process, which is set as
J= T
0
acRdt, (31)
wherecR($/m2s)is the price of rent, including operational costs. The spacing and plant density are connected via
v=aW. (32)
Thus plant density W can be considered as a control variable instead of the spacing a. The state equation and the expres-sion for the performance index can be rewritten in the form
dv dt =
v WG(W), J =
T
0
v
WcRdt. (33)
The final time T is free and the final moment T is determined through the condition
v(T )=vT, (34)
where vT is a given final marketable weight of a single lettuce plant. The Hamiltonian, H, has the form
H= v
W(pG(W)−cR), (35)
where p is the scalar costate variable. The maximization of the Hamiltonian over the control W gives
jH
jW = − v
W2(pG(W)−cR)+
vp W
jG
jW =0. (36)
From here the value of p is found as
p= cR
G(W)−WjjWG. (37)
The differential equation for the co-state p has form dp
dt = −
jH
jv = −
pG(W)−cR
W . (38)
Substituting an equation for p from (37) to (38) one gets after some transformations
dp dt = −
cRjjWG
(G−WjjWG). (39)
Differentiating (37) with respect to t one gets
dp dt =
dW dt
cRWjG 2
jW2
(G−WjjWG)2. (40)
From (39) and (40) one gets the differential equation for the control variable W as follows
dW dt = −
jG
jW(G−WjjWG)
WjjWG22
. (41)
For the free final time we have the transversality condition
H (T )=0. (42)
Substituting the expression for p (37) and the expression for
H (35) into the Eq. (42) one gets for the final moment T the
following equality
v cR jG
jW
(G−WjjWG)=0. (43)
From (43) and (41) it follows that there exists an optimal steady-state solution with the constant control value W∗ which satisfies the condition
jG(W)
jW =0. (44)
Thus, the optimal lettuce density is constant and corresponds to the maximal net photosynthesis. The problem is solved without maximization of the Hamiltonian, which in this case seems rather simple, but unexpected in advance. However, if the final time is not free, and is fixed a priori at some non-optimal value tf, Eq. (41) makes is possible to avoid the maximization of the Hamiltonian, which constitutes a difficult numerical problem for each time step. For details, see Ioslovich and Gutman, 1999.
4. Maximal area surrounded by a curve of given length with fixed end points on a horizontal axis
This is a very well-known problem described in e.g.
problem. We shall solve it in the case of given parameters as an OCP
1 0
x1dt→max, (45)
dx1
dt =u, (46)
dx2
dt =
1+u2. (47)
The considered end conditions are
x1(0)=x1(1)=0, x2(0)=0, x2(1)=/3. (48)
The Hamiltonian has the form
H=p1u+p2
1+u2+x
1. (49)
The differential equations for the costate variables are dp1
dt = −1, dp2
dt =0. (50)
From (49) one gets
jH
ju =p1+p2√ u
1+u2=0. (51)
In fact (51) is possible to solve with respect to u. However this is not always the case in general, and we want to demon-strate our new method. Here we usep1aspaandp2aspb.
From (51) the value ofp1is found in the form
p1= −p2√ u
1+u2. (52)
Taking the derivative with respect to t of (52) and using (50) one gets
du dt =
(1+u2)3/2
p2 .
(53)
Now, taking into account thatp2is a constant according to
(50), we have to integrate numerically Eqs. (46), (47), and (53) in order to find the unknown constantsp2andu(0)such
that conditions (48) are satisfied. The answer isp2= −1,
andu(0)=tan/6=√1
3.The resulting curvex1(t)is a part
of a circle. As in the other examples, we did not have to solve the non-linear equation (51).
This problem also can be solved analytically as follows. One can divide (53) over (47) and get
du dx2 =
√
1+u2
p2 ,
(54)
thus changing independent variable from t tox2. Eq. (54)
can be solved and be given as
arctanu= x2
p2 +C,
(55)
where C is an unknown integration constant. From (55) one gets
u=tan(x2/p2+C). (56)
From here we can get
1+u2=1/cos(x
2/p2+C).
Now one can integrate (47) and get
sin(x2/p2+C)=(t+C1)/p2, (57)
where C1 is another integration constant. Doing the same
substitution of x2 instead of t in (46), (47) as above and
using (56), one gets dx1
dx2=
sin(x2/p2+C). (58)
From here it follows
(C2−x1)/p2=cos(x2/p2+C), (59)
whereC2again is an additional integration constant. From
(57) and (59) it follows
(x1−C2)2+(t+C1)2=p22, (60)
which means that the optimal curve is a part of a circle. Unknown constantsp2, C, C1, C2are easily found from the
end conditions fort, x1, x2and we get
p2= −1, C=/6, C1= −1/2, C2= − √
3/2, from which it follows from (56) thatu(0)=tan/6=1/√3.
5. Conclusion
A method for extracting smooth optimal control for a class of optimal control problems was presented and three illus-trative examples were shown. The method does not require the maximization of the Hamiltonian over the control, and instead substitute the ODE for the part of costates by the ODE for the smooth control. One could note that the method has advantage only if the equations jH/ju=0 cannot be solved analytically.
Acknowledgements
The authors are grateful to Professors V.F. Krotov, A. Ioffe, and B.T. Polyak for fruitful discussions. The authors also want to thank the Associate Editor and anonymous reviewers for useful notes.
References
Gelfand, I. M., & Fomin, S. V. (1969). Calculus of variations (p. 228). Englwood Cliffs, NJ: Prentice-Hall.
Ioslovich, I., & Gutman, P. -O. (1999). Plant factory optimization. Third international conference on industrial automation Montreal, Canada; international Association for Industrial Automation (IAIA), Conference Proceedings (pp. 15.5–15.8).
Ioslovich, I., & Gutman, P. -O. (2000). Optimal control of crop spacing in a plant factory. Automatica, 36, 1665–1668.
Lee, E. B., & Markus, L. (1967). Foundations of optimal control theory (p. 576). NY: Wiley.
Pinch, E. R. (1993). Optimal control and the calculus of variations (p. 234). UK: Oxford Science Publication.
Pontryagin, L., Boltyansky, V., Gamkrelidze, R., & Mischenko, E. (1962). Mathematical theory of optimal processes (p. 383). NY: Wiley. Seginer, I., & Ioslovich, I. (1999). Optimal spacing and cultivation
