188
Optimal Control and Problem with Linear Quadratic Cost
Function
Habib Molaei
Department of Mathematics Technical and Vocational, University Gazi-Tabatabaei Urmia, Iran, P.O. Box 57169-33950
Abstract
In this paper, we explore exactly this approach for the linear quadratic optimal control problem. We will consider how to stabilize linear system by using feedback control. This topic is very useful in physics, Biology and Engineering. We consider the linear Quadratic optimal control and introduce the riccati differential equation and riccati algebraic equation. We look at the optimum of a function and functional and some variational problems. Various steps used in finding the optimal solution to these variational problems discussed. We also consider the exterma of functions and functionals with conditions. Also, Examples on finding the exterma were given with their solutions.
Mathematics subject classification: 49J15, 49K21, 34B15
Keywords:optimal control, Boundary conditions, Lagrangian function, Linear quadratic, Cost function
1. Introduction
Consider minimization of a general performance index for an arbitrary initial point (𝑥,t)
J( u) =φ( 𝑥(𝑡𝑓),𝑡𝑓) +∫𝑡𝑡0𝑓V(𝑥(t), u(t), t)dt (1)
Subject to the following system with final time constraints:
𝑥̇(𝑡)=F(𝑥 (t),u(t),t) , tϵ[𝑡0,𝑡𝑓] (2) 𝑥 (tR0R)=𝑥0 (3) 𝑥 (tR0R) , 𝑡𝑓is free (4)
Here, 𝑥ϵRn, tϵR, uϵRm and φ is functional of 𝑥(t),V is a functional of 𝑥(t),u(t)
We assume that there exists no constraints on the state and control trajectiories. Our objective is to evaluate the optimal trajectory statisfying the final time constrains and to find the optimal feedback control for an arbitrary initial point (𝑥,t)ϵRP
n+1
P
. We consider two representative problem formulations, which are characterized by the types of terminal boundary conditions:
i. Hard constraint problem (HCP): φ�𝑥�𝑡𝑓�,𝑡𝑓�= 0 , φ�𝑥�𝑡𝑓�,𝑡𝑓�=𝑥(𝑡𝑓)− 𝑥𝑓 .
ii. Soft constraint problem (SCP): φ�𝑥�𝑡𝑓�,𝑡𝑓� dose not exit and φ�𝑥�𝑡𝑓�,𝑡𝑓�ϵR.
2. Variational approach to optimal control system
Equivalent to the a love equation is as follows: Find the extrema of
J(u) =∫ �𝑡𝑓 V(𝑥(t), u(t), t) +dφdt�dt
t0 (5)
Subject to
𝑥̇(𝑡) =𝑓(𝑥(𝑡),𝑢(𝑡),𝑡) (6)
𝑥 (tR0R)=𝑥0 (7) 𝑥 (𝑡𝑓 ), 𝑡𝑓 is free (8)
Solution
Step1: Introduce the Lagrangian function L i.e
L(𝑥,𝑥̇,𝑢,𝜆,𝑡) =�V(𝑥(t), u(t), t) +dφdt�=λ(t)(f(𝑥(t), u(t), t)− 𝑥̇(𝑡)) (9)
189
𝐽𝑎=𝐽𝑎�𝑥,𝑥̇,𝑢,𝜆,𝑡𝑓�=∫ 𝐿𝑡𝑡0𝑓 (𝑥,𝑥̇,𝑢,𝜆,𝑡)𝑑𝑡 (10)
Step2:Find the first variation given increments Δ𝑥,∆𝑥,̇ ∆𝑢 ,∆𝑡𝑓
∂𝐽𝑎
∂𝑥 =0, ∂𝐽𝑎
∂𝑥̇ =0 , ∂𝐽𝑎
∂u =0 , ∂𝐽𝑎
∂λ =0 (11)
𝐽𝑎(𝑥+Δ𝑥 ,𝑥̇+∆𝑥̇, u +Δu, tf+λtf)− 𝐽𝑎(𝑥,𝑥̇, u, tf) =
�𝑡𝑓+∆𝑡𝑓L
𝑡0
(𝑥+Δ𝑥,𝑥̇+∆𝑥̇, u +Δu,λ)dt− � 𝐿𝑡𝑓 (𝑥,𝑥̇,𝑢,𝜆,𝑡)𝑑𝑡
𝑡0
=
∫tf[L
t0 (𝑥+Δ𝑥,𝑥̇+∆𝑥̇, u +Δu,λ)−L(𝑥,𝑥̇,𝑢,𝜆)]dt +∫ L(𝑥+Δ𝑥,𝑥̇+∆𝑥̇
tf+Δtf
tf , u +Δu,λ)dt (12)
By Taylor series expansion
=∫tf(∂𝑥∂l
t0 Δ𝑥+
∂l ∂𝑥̇Δ𝑥̇+
∂l
∂𝑢Δu + oh(p))dt + L(𝑥,𝑥̇, u,𝜆)�
Δ𝑡𝑓
t =𝑡𝑓+ oh
(p) (13)
Using integration by parts
∫𝑡𝑓𝜕𝑥̇𝜕𝑙𝛥𝑥̇
𝑡0 𝑑𝑡=∫
𝜕𝑙 𝜕𝑥̇ 𝑡𝑓
𝑡0 𝑑(𝛥𝑥) =
𝜕𝑙 𝜕𝑥̇𝛥𝑥 �
𝑡𝑓
t0−∫ (∆𝑥) d dt 𝑡𝑓
t0 (
𝜕𝑙
𝜕𝑥̇)dt (14)
Equation (13) becomes
=∫𝑡𝑓(∂𝑥∂l
t0 −
d dt(
𝜕𝑙
𝜕𝑥̇))∆𝑥dt +∫ 𝜕𝑙 𝜕𝑢𝛥𝑢 𝑡𝑓
𝑡0 𝑑𝑡+ L(𝑥,𝑥̇, u,λ)�
Δ𝑡𝑓
t =𝑡𝑓+ 𝜕𝑙 𝜕𝑥̇Δ𝑥 �
𝑡𝑓
t0+ oh
(p) (15) The necessary condition for the extrema point are
∂l ∂𝑥−
d dt�
𝜕𝑙
𝜕𝑥̇�= 0 (16)
∂l ∂u = 0 L(𝑥,𝑥̇, u,λ)� Δ𝑡𝑓
t =𝑡𝑓+ ( 𝜕𝑙 𝜕𝑥̇Δ𝑥)�
𝑡𝑓
𝑡0=0
∂l ∂λ= 0
Step3: Introduce Hamiltonian function H(𝑥, u,λ, t)
H(𝑥, u,λ, t) = V(𝑥, u, t) +λ(t)f(𝑥, u, t) (17) But L = V(𝑥, u, t) +dφ
dt+λ(t)(f(𝑥, u, t)− 𝑥̇)
L = H(𝑥, u,λ, t) +dφdt− λ(t)𝑥̇(𝑡) (18)
∂l ∂𝑥− d dt� 𝜕𝑙 𝜕𝑥̇�= ∂H ∂𝑥+ ∂ ∂𝑥� 𝑑𝜑 𝑑𝑡� −0−
d dt(
∂ 𝜕𝑥̇�
𝑑𝜑
𝑑𝑡� − λ(t)) (19)
Note that
dφ dt =
dφ(𝑥(𝑡),t)
dt = ∂φ(𝑥,t)
∂x 𝑥̇+ 𝜕𝜑(𝑥,𝑡)
𝜕𝑡 (20)
We have ∂l ∂𝑥 − d dt� 𝜕𝑙 𝜕𝑥̇�= ∂H ∂x+
∂2φ(x, t)
∂x2 𝑥̇+
∂2φ(x, t)
∂x∂t − d dt�
∂φ(x, t)
∂x − λ(t)� =∂H∂𝑥+∂2∂𝑥φ(2x,t)𝑥̇+
∂2φ(x,t)
∂x ∂t − � 𝜕2𝜑(𝑥,𝑡)
𝜕𝑥2 𝑥̇+
𝜕2𝜑(𝑥,𝑡)
𝜕𝑥𝜕𝑡 − 𝑑𝜆 𝑑𝑡�= ∂H ∂𝑥+ dλ
dt= 0 (21)
From (18)
∂l ∂u=
∂H
∂u=0 (22) ∂l
∂λ= ∂H
∂λ− 𝑥̇(𝑡) = 0 (23)
Step 4:
The necessary conditions are
∂l ∂𝑥−
d dt�
𝜕𝑙
𝜕𝑥̇�= 0⟹ ∂H
∂𝑥 =−𝜆̇(𝑡) (24) ∂l
∂u= 0⟹ ∂H
190
L(𝑥,𝑥̇, u,λ)�t =Δ𝑡𝑓𝑡𝑓+ 𝜕𝑙 𝜕𝑥̇Δ𝑥 �
𝑡𝑓
t0=0 (26)
⟹ �H +∂φ∂t� �t =Δ𝑡𝑓𝑡
𝑓+� ∂φ
∂𝑥− λ(t)� �
Δ𝑥𝑓
t =𝑡𝑓 = 0 (27) ∂l
∂λ= 0⟹ ∂H
∂λ=𝑥̇(𝑡) (28)
Example
Find the extrema (i.e the optimal control and optimal state)
J(u) =12�𝑡𝑓u2 𝑡0
(t)dt
Under the conditions 𝑥̇1(t) =𝑥2(t) , 𝑡0= 0
𝑥̇2(t) = u(t) , 𝑡𝑓 = 2 �𝑥𝑥1
2�(t0) =�
1 2� �𝑥𝑥1
2� �𝑡𝑓�=�
1 0�
Solution
From the problem, we identify the following
𝑥=�𝑥𝑥12� , V(𝑥, u, t) = V�u(t)�=12 u2(t)
f(𝑥, u, t) =�ff1
2� Where f1=𝑥2 , f2= u
Step 1:
We formulate the Hamiltonian function as
H = H(𝑥1(t),𝑥2(𝑡), u(t),λ1(t),λ2(t)) = V +λf
=12 u2(t) +�λ
1,λ2� �𝑥2u �
=12 u2(t) +𝜆
1𝑥2 +λ2u
Step2:
Find u from
∂H
∂u = 0⟹u +λ2= 0⟹u =−λ2
Step3:
Using the results of step2 in step1, find the optimal
H =1
2λ22+λ1𝑥2− λ22 =λ1𝑥2−12λ22
Step4:
Obtain the state and costate equations from
H(𝑥1,𝑥2,λ1,λ2) =λ1𝑥2−12λ22
∂H
∂𝑥1= 0 ,
∂H
∂𝑥2=λ1
∂H
∂λ1=𝑥2 ,
∂H
∂λ2=−λ2
⎣ ⎢ ⎢
⎡∂H
∂𝑥1
∂H
∂𝑥2⎦
⎥ ⎥ ⎤
+dtd�λ1
λ2�= 0
�λ0
1�+�
dλ1
dt dλ2
dt
191
λ1+ddt = 0λ2 →c1+ddt = 0λ2 ⟹ λ2=−c1t + c2
∂H
∂λ2=−λ2= u
𝑥̇2=λ2⟹ 𝑥̇2=−(−c1t + c2) = c1t−c2
d𝑥2
dt = c1t−c2
𝑥2=c1t
2
2 −c2t + c3
𝑥̇1=𝑥2
𝑥̇1=c1t
2
2 −c2t + c3⟹ 𝑥1= c1t3
6 −
c2t2
2 + c3t + c4
Step5:
Obtain the optimal control from
u =−λ2
u = c1t−c2
Where c1, c2, c3 and c4 are constants evaluated using the boundary conditons given that is
�
𝑐1𝑡3
6 − 𝑐2𝑡
2
2 +𝑐3𝑡+𝑐4
𝑐1𝑡2
2 − 𝑐2𝑡+𝑐3
�(𝑡0) =�12�
�cc4
3�=�
1
2� ⟹c4= 1, c3= 2
Also
�
c1t3
6 −
c2t2
2 + c3t + c4
𝑐1𝑡2
2 − 𝑐2𝑡+𝑐3
�(𝑡𝑓) = �10�
�
8c1t3
6 −
4c2t2
2 + 2c3t + c4 4𝑐1
2 −2𝑐1+𝑐3
�= �10�
Solving these equations simultaneously gives c2= 4 , c1= 3 Finally, we have the optimal states constates and control as
𝑥1(t) = 0.5t3−2t2+ 2t + 1
𝑥2(t) = 1.5t2−4t + 2
λ1(t) = 3
λ2(t) =−3t3+ 4
u(t) = 3t−4
J(u) =12� (3t−4)2dt =1
2
2
0 [24 + 32−48] = 4
3. Linear Quadratic
In this section we explore exactly this approach for the linear quadratic optimal control problem. Consider the optimal control for arbitrary for the system
𝑥̇= a𝑥+ bu (29) Where 𝑥= R is a scalar state , uϵR is the input, the initial state 𝑥(𝑡0) is given and a, bϵR are positive constants.
We wish to find a trajectory (𝑥(t), u(t)) that minimizes the cost function.
J(u) =12∫ u2(t)dt +1 2 𝑡𝑓
𝑡0 c𝑥
2(𝑡
𝑓) (30)
Where the terminal time 𝑡𝑓is given and c > 0 is a constant. This cost function balances the final value of the state with the input required to get to that state.
To solve the problem, we define:
192
φ=12c𝑥2(𝑡𝑓) (32)
We write the Hamiltonian of this system and derive the following expressions for the costate λ:
H = V +λf =12u2+λ(a𝑥+ bu) (33)
𝜆̇=−∂H∂x=−aλ , λ�𝑡𝑓�=∂φ∂𝑥= c𝑥�𝑡𝑓� (34)
This ia a final value problem for a linear differential equation in λ and the solution can be shown to be
λ(t) = c𝑥�𝑡𝑓�ea(𝑡𝑓−t) (35)
The optimal control is given by
∂H
∂u= u + bλ= 0⟹u∗(t)−bλ(t) =−bc𝑥(𝑡𝑓)ea(𝑡𝑓−t) (36)
Substituting this control into the dynamics given by equation (29) yields a first-order ODE in 𝑥:
𝑥̇= a𝑥 −b2c𝑥(𝑡
𝑓)ea(𝑡𝑓−t) (37)
This can be solved explictiy as
𝑥∗(t)=𝑥(t 0)ea
(t−t0)
+b2a2c𝑥∗(𝑡
𝑓)[ea(𝑡𝑓−t)−ea(t+𝑡𝑓−2t0)] (38)
Setting t =𝑡𝑓 and solving for 𝑥�𝑡𝑓� ,Gives
𝑥∗�𝑡
𝑓�= 2ae
a(𝑡𝑓−t0)𝑥(t0)
2a−b2c(1−e2a(tf−t0)) (39) And finally we can write
u∗(t) =−2abc ea(2tf−t0−t)𝑥(t0)
2a−b2c(1−e2a(tf−t0)) (40) 𝑥∗(t) =𝑥(t
0)ea(t−t0)+ b
2c ea(tf−t0)𝑥(t
0)
2a−b2c�1−e2a(tf−t0)�[ea(tf−t)−ea(t+tf−2t0) (41)
We can use form of this expression to explore how our cost function effect the optimal trajectory. For example, we can ask what happens to the terminal state x∗�𝑡𝑓� and c→ ∞ .
Setting t =𝑡𝑓 in equation (41) and taking the limit we find that
limc→∞𝑥∗(tf) = 0 (42)
4. Conclusions
In this work we can use expression to explore how our cost function effects the optimal trajectory, also using these tools we derive the linear quadratic regulator for linear systems and describe its usage.
Acknowledgements
Author of contributors to this paper helpful comments thanks.
References
[1] Anderson,B.D.O. and J.B. Moore, Optimal Control, Linear Quadralic Methods, prentice-hall, Englewood C liffs, NJ, 1989.
[2] M.E.J. Newman. M. Girvan, and J.D. Farmer, Optimal Design. Robustness and Risk Aversion, phys. ReV. Lett.,89,028301(2002)
[3] Habib.H.Molaei, Gradient in Optimal Control Problem With Non-local Boundary Conditions. Transactions of NASA, Series of Mathematical science and technical, Baku, XXVI, No 7 (2006).
[4] Molaei H.H. Difference Approximation of Quadrature Optimal Control Problem With Integral Conditions. The 1-st International with Industrial Applications, Baku, Azerbaijan, May 22-25,2006, pp73-74.
