The familiar Euler-Lagrange (E-L) theory transforms the problem of optimizing a cost function into a well-defined two-point boundary value problem (TPBVP). Orig-
inating from the calculus of variations, Lagrange developed the approach in response to the challenges of the brachistochrone problem. The E-L Theorem provides a set of necessary conditions to be satisfied that enable the optimal control, defined in terms
of the control vector, u∗(t), to minimize the scalar cost function, J . The number
of these conditions is sufficient to ascertain a unique local solution to the TPBVP, thus, it is denoted as “well-defined”. The E-L theorem is an indirect optimization ap- proach because an optimal solution satisfies a set of boundary conditions rather than one that directly minimizes the cost function. A solution that satisfies the necessary conditions of the E-L theorem is guaranteed to be a locally extremal solution, and additional rules are available to verify that the solution is a minimum or a maximum. Greater detail on the derivation and application of the E-L theory are available in the text Optimal Control with Aerospace Applications by Longuski, et al. [6].
An optimization problem is distinguished from a general trajectory design problem by the inclusion of an objective function, this expression incorporates the quantities of importance that determine the optimality of a solution. The objective function, or cost function, produces a scalar value, J , to be minimized,
Min. J = φ(tf, xf) | {z } Terminal Cost + Z tf t0 L(t, x, u)dt | {z } Path Cost (4.1)
The integrand, L, is termed the Lagrangian and is dependent upon m control vari- ables, compromising the vector u, in addition to n states, that is, x and time, t. The cost function in equation (4.1) is generally formulated as a problem of Bolza, con- taining both terminal and path components of the cost. A cost function consisting only of a terminal cost is in a Mayer form while a cost function with only a path component is in Lagrange form.
Application of the E-L Theorem to an optimal control problem is divided into a series of three distinct steps. The first of these steps is forming the Hamiltonian, i.e.,
where f (t, x, u) = ˙x is compromised of the n system scalar equations and the λ vector contains Lagrange multipliers also called costates. The second step in the process of applying the E-L theorem to an optimal control problem is to evaluate the partial derivatives of the Hamiltonian according to the E-L necessary conditions. This computation results in both differential and algebraic equations,
˙
λ = ∂H
∂x = Hx (4.3)
∂H
∂u = Hu= 0 (4.4)
Equation (4.3) requires a partial derivative with respect to each of the n state vari- ables, therefore, this necessary condition results in n differential equations for the costates. Equation (4.4) produces m algebraic equations from the partial derivatives of H with respect to the control variables. A TPBVP is formed from the application of the first two steps of the Euler-Lagrange theory to the optimal control problem, however the resulting problem may be ill-defined due to an insufficient number of boundary conditions.
The third step in the process of applying the Euler-Lagrange theory to an optimal control problem supply additional boundary conditions. An optimal control problem includes a set of p boundary conditions that define necessary initial or final conditions for the problem, and these are collected into a vector such that,
ψ(t, x) = 0 (4.5)
To pose the optimal control problem as a well-defined TPBVP, a total of 2n + 2 boundary conditions are required. The initial number of boundary conditions rarely satisfies this requirement, therefore, additional conditions are typically added. The initial conditions for the state variables and time supply n + 1 conditions; therefore, the necessary number of additional conditions is 2n + 2 − (n + 1) − p = n + 1 − p. To derive the n + 1 − p boundary conditions for a well-defined TPBVP, the terminal boundary constraints are first differentiated, i.e.,
The known boundary conditions are substituted into equation (4.6), producing p inde- pendent equations expressed in terms of the n+1−p remaining unknown differentials. The p independent equations are substituted into the transversality condition that initially includes n + 1 differential terms,
Hfdtf − λTfdxf + dφ = 0 (4.7)
Following this substitution, the transversality condition contains n + 1 − p terms. The coefficients of each term in equation 4.7 are set to zero and these n + 1 − p expressions supply the final boundary conditions to complete the required number. The result of these operations is 2n + 2 boundary conditions that allow a well-defined TPBVP to be formulated. This method for deriving the additional boundary conditions required for the TPBVP is termed the un-adjoined method and is one of two approaches. The alternate method involves the introduction of extra adjoining variables and, hence, is denoted the adjoined method. The usage of the adjoined technique has increased since its inclusion in several key texts such as Bryson and Ho [8] in 1969 and it is the predominate method taught in modern textbooks [32]. However, both methods are presented by Citron [33] and Longuski [6]. The popularity of the adjoined method is largely due to the ease with which it is implemented in various types of numerical methods, which are required to solve all but the most elementary of optimal control problems.
Application of the E-L theorem alone is sometimes insufficient to fully solve the optimal control problem. In such circumstances, the minimum principle is also ap-
plied to determine a control law that determines u∗(t) and, thus, minimizes the cost
function, J . The most general form of this condition is Pontryagin’s Minimum Prin- ciple [34], often simply identified as the Minimum Principle,
H[t, x∗(t), u∗(t), λ(t)] ≤ H[t, x∗(t), u(t), λ(t)] (4.8)
Bryson and Ho [8] offer McShane’s succinct summary of the Minimum Principle as:
equation (4.8) include a control vector that is continuous or piecewise as well as bounded or unbounded. The Weierstrass necessary condition is a similar, although more restrictive version of the Minimum Principle that applies only to scenarios with unbounded continuous control. However, many concepts in spacecraft trajectory op- timization require piecewise bounded control, for example, the “bang-bang” control history of a constant specific impulse low-thrust spacecraft, therefore, the general Minimum Principle is more frequently applied. Finally, if a maximum value of the cost function is sought, the inequality in equation (4.8) is reversed to produce the cor- responding Maximum Principle. Application of the Minimum or Maximum Principles ensures that the desired type of extremal is obtained when the TPBVP is solved.
The E-L algebraic necessary condition in Equation (4.4) is only applicable to optimal control problems with unbounded control. When the value of a scalar control
variable u is bounded, for example |u| ≤ umax, then the partial derivative of the
Hamiltonian with respect to the control may not be zero, rather, ∂H
∂u = H1 (4.9)
where the variable H1 corresponds to a Hamiltonian assumed to be linear with respect
to u, that is,
H = H0(t, x, λ) + H1(t, x, λ)u (4.10)
The sign of H1 is employed, in combination with the minimum principle, to determine
a value for u. According to the maximum principle u is such that H is maximized, thus the following rule guides the choice of u.
u = −umax if H1 < 0 undetermined if H1 = 0 umax if H1 > 0 (4.11)
If an optimal control problem is formulated to minimize a given objective, then the minimum principle applies and the first and last elements of equation (4.11) are reversed. Note that when a bounded control variable is a vector the approach just
described for scalar control values applies to each component of the vector variable. Therefore, algebraic equations for the control variables can be derived even when the control is bounded.