6.3 A Primal–Dual Parametric Active Set Strategy
6.3.1 Active Set Methods
Active set approaches usually are evaluated under the following aspects concerning their ef- ficiency, and in this section we present a parametric primal–dual active set method due to [25, 67] for the solution of parametric QPs that is notably distinguished from more classical active set methods in several points.
Finding a Feasible Point
This usually is the first requirement to start an active set method. Between one third and one half of the total effort of solving a QP is typically spent for finding an initial feasible guess [88]. The task of identifying such an initial guess is usually referred to as phase one. Several different approaches exist, such as the solution of an auxiliary Linear Program (LP) or the relaxation of constraints in a homotopy, cf. [157].
In SQP methods and in Model Predictive Control (MPC), a series of closely related QPs is solved. For each QP, the preceding QP’s solution is an obvious candidate for a feasible point, a warm starting technique. Changes to the constraint matrix A or vector b may render this candidate infeasible, though, mandating a phase one run. Our active set algorithm is able to start in any predefined primal point.
Determining the Active Set Exchange
This action is necessary whenever more than one component of the dual vector λ indicates non–optimality. Different pivoting strategies such as choosing the constraint with the lowest index (lexical pivoting) or with the most negative value λiare in use. Our algorithm adds and removes constraints in the order they appear on the homotopy path, thereby avoiding the need for pivoting strategies.
Ensuring Linear Independence of the Active Set
Whenever a constraint is added to the active set, linear dependence must be verified and its maintenance may require removing another active constraint. There may in addition be degenerate points in which the full active set is linearly dependent and thus will not be fully identified. Repeated addition and removal of constraints may happen with steps of zero length in between if such a point is reached, a situation that has already been investigated from the NLP point of view in section 5.2. Ultimately, the method may revisit an active set without having made significant progress in the primal–dual iterate (x, λ). The method is said to be cycling, possibly infinitely.
Depending on the class of QPs treated, the cycling phenomenon is either ignored or treated by simple heuristics. The EXPAND strategy [85] used by many codes [73, 84, 86] perturbs constraints in order to ensure progress in the primal–dual variables, but was shown to fail on specially crafted instances [98]. Our active set method monitors positive progress on the homotopy path and can easily detect and resolve linear dependence. In the case of ties, cycling may occur and resolution techniques are presented by [215].
KKT Solution and Matrix Updates
Solving the EQP to find the primal–dual step involves a factorization and backsolve with the KKT matrix associated with the current active set. Computing this factorization is necessary after each change of the active set, and usually comes at the cost of O(n3) floating–point operations where n denotes the number of unknowns in the QP. Exploiting structure and sparsity is crucial for the efficiency of the active set method. For certain factorizations, it is possible to compute the factorization only once and recover it in O(n2) operations by using matrix updates after each active set change. These issues are considered in chapters 7 and 8.
6.3.2 A Primal–Dual Parametric Active Set Strategy
We now present a parametric active set method due to [25, 67] that computes optimal solu- tions (x?(τ), λ?(τ))def
= (x?(p(τ)), λ?(p(τ))) of a PQP along a homotopy path between given parameters p0, p1∈ P,
p(τ)def
= p0+ τδp, δp
def
= p1− p0, τ ∈ [0, 1] ⊂ R, (6.16)
given the optimal solution (x?
(0), λ?(0)) of the PQP in τ = 0, i.e., for the initial parameter p0 ∈ Rnp
. Gradient and residual can be reparameterized as functions of the scalar homotopy parameter τ,
b(τ)def
= b0+ B(p0+ τδp), g (τ)
def
= g0+ G(p0+ τδp). (6.17) In addition, the distance to the homotopy end point is written as
δb(τ)= b(1) − b(τ) = (1 − τ)Bδp,def δg (τ)def= g (1) − g (τ) = (1 − τ)Gδp. (6.18)
Rationale
The basic idea of the parametric active set strategy to be presented is to move along the homotopy path from the known solution in τ = 0 to the solution sought in τ = 1 while main- taining both primal and dual feasibility, i.e., optimality of the iterates. This is accomplished by computing homotopies
x?
(τ) : [0, 1] → Rn, (6.19)
λ?(τ) : [0, 1] → Rm,
W(τ) : [0, 1] → {1, . . . , m} ⊂ N,
that satisfy the KKT theorem 3.1 in every point τ ∈ [0, 1] ⊂ R and start in the known point (x?(0), λ?(0)) with the working set W(0) = W(x?(0)). Regularity of the KKT matrix implies that x?(·) and λ?(·) are piecewise affine and that x?(·) in addition is continuous by theorem 6.6. Hence there exist k¾ 2 primal–dual points (xi, λi) ∈ Rn+m which for 2¶ i ¶ k − 1 are located on the common boundaries of pairs (Ci−1, Ci) of critical regions, such that
x?
(τ) = xi+ (τ − τi)δxi, x (τ1) = x?(0), (6.20)
λ?(τ) = λi+ (τ − τi)δλi, λ(τ1) = λ?(0), ∀τ ∈ [τi, τi+1], 1 ¶ i ¶ k − 1, x (τi+1) = x (τi) + (τi+1− τi)δxi.
Figure 6.2 depicts this situation in the space P of feasible homotopy parameters.
A Primal–Dual Iteration
In the first iteration k = 1, the algorithm starts for τ1 = 0 in the given optimal solution (x1, λ1) = (x?(0), λ?(0)). The initial working set W1 is chosen as a linear independent subset of the active set A(x1).
p0 τ1= 0 τ2 τ3 τ4 τ5= 1 p1 C1 C2 C3 C4
Figure 6.2: The piecewise affine homotopy path.
linear system H AT Wk AWk −δx δλWk = δg (τ k) −δbWk(τk) (6.21)
for the point τk∈ [0, 1] ⊂ R on the homotopy path. Appropriate techniques are investigated in chapter 7. The maximum step length maintaining primal feasibility is found from
αkprim= mindef ¨ bi(τk ) − Ai?x Ai?δx− δbi(τk) i /∈ Wk ∧ Ai?δx− δbi(τk) < 0 « . (6.22)
Constraints i satisfying this minimum are called primal blocking constraints. Likewise the max- imum step maintaining dual feasibility is found from
αkdual= mindef ¨ −δλλj j j ∈ W k ∧ δλ j< 0 « , (6.23)
and constraints j satisfying this minimum are called dual blocking constraints. By choosing
αkdef= minn1 − τk, αkprim, αdualk o
(6.24)
we move onto the closest blocking constraint on the remainder [τk, 1] of the homotopy path, xk+1def
= xk+ αkδx , λk+1= λdef k+ αkδλ, τk+1= τdef k+ αk. (6.25) To find the new working set, we distinguish the following three cases:
1. An inactive constraint i enters the working set if τk+1= τk
+αkprim: Linear independence of the new working set can be checked by solving
H AT Wk AWk v w = a 0 (6.26)
where aT = Adef i?is the constraint row i of A entering the working set. If v 6= 0 the new working set Wk+1def
= Wk∪{i} is linear independent. Linear dependence of the active set is indicated by v = 0. If then w ¶ 0, the PQP is infeasible for all τ ¾ τk+1. Otherwise, by removing the constraint
j = argmin ¨λk+1 j wj j ∈ W k ∧ w j> 0 « (6.27)
from the active set we can restore linear independence of Wk+1 def
= Wk∪ {i} \ {j}. In addition, we set
λk+1i = λdef k+1j /wj, λk+1j = 0,def λk+1Wk
def
= λWk+1k − λk+1i wWk. (6.28)
2. An active constraint j leaves the working set if τk+1= τk+ αkdual: Boundedness of the new EQP can be checked by solving
H AT Wk AWk v w = 0 −ej . (6.29)
If w 6= 0, boundedness is maintained. Semidefiniteness of the Hessian is indicated by w = 0. If then Av ¾ 0, the PQP step is unbounded for all τ ¾ τk+1. Otherwise, by adding the constraint
i = argmin ¨ bi(τk+1) − Ai?x(k+1) Ai?v i /∈ Wk ∧ A i?v < 0 «