Remark 7 (Termination). Algorithm 5 terminates when a prespecified accuracy is reached in
terms of the progress of the dual sequence. The extracted primal sequence is given by
(xk,uk) = arg min
u,x {L(u, x, λ
k|Tk)| x
i+1 = Axi+ Bui, x0= xinit} (3.11)
for i ∈ 0, . . . , Tk and xki = (A + BKLQ)i−TkxTk for i > Tk. In Theorem 5 below it is proven
that (xk,uk) tends to the optimal constrained LQ solution. At any finite iterate, however, the
sequence (xk,uk) may violate the constraints. In order to remedy this we solve upon termination an
equality-constrained QP where we minimize the objective function subject to the active constraints at optimality. The active constraints can be (approximately) detected by looking at the nonzero values of the dual vector λk at termination. This step comes at a very small cost since it involves one solution of a KKT system of linear equations.
3.6
Convergence results
In the previous section we gave an implementable algorithmic scheme that computes the solution to the CLQR problem. Here we provide all the necessary proofs which allow us to assert that the solution generated by Algorithm 5 via (3.11) indeed converges to the true optimizer of the CLQR problem. In what followsλ∞denotes any optimal solution to the dual problem (3.6) (which exists under Assumption 1 but may not be unique) and (u∞,x∞) the optimal solution to the primal problem (3.1). Our main result is:
Theorem 5 (Main Theorem). Suppose Assumption 1 holds and letλkbe a sequence of iterates generated by Algorithm 4 and (xk,uk) the associated primal sequence given by (3.11) and let L be
a Lipschitz constant of ∇h. The following statements hold:
1. The composite function F (λ) = h(λ) + δ−(λ) as defined in (3.6) converges as
F (λk)− F (λ∞)≤ a
2L
2(k + a− 1)2λ0− λ∞2Hλ .
2. The sequence of the dual iterates{λk} converges weakly (see Definition 17 in Appendix 3.10.1) to an optimizer, that is,
λk λ∞
for someλ∞∈ arg min F .
3. The input sequence {uk} converges strongly to the unique minimizer as uk− u∞H u ≤ a & L μ λ0− λ∞H λ (k + a− 1) ,
4. The state sequence{xk} converges strongly to the unique minimizer as xk− x∞H x ≤ a & B2L μ λ0− λ∞H λ (k + a− 1) .
5. The sequence {Tk} is bounded.
Proof: 1. Convergence of F (λk) with a constant stepsize is proven in [CD15, Theorem 1]. Convergence at the same rate with an adaptive stepsize generated from the backtracking Algorithm 6 is proven in Lemma 11, Appendix 3.10.3.
2. The proof is stated in [CD15, Theorem 3].
3. The idea is to upper bound the input sequence’s convergence rate making use of the first point’s result. In order to do so we make use of strong duality. The proof is inspired from [BT14, Theorem 4.1] and is as follows:
Let λk ≤ 0 generated from Step 7 of Algorithm 5. Denote uk = arg min
u∈Hu
f(u) := f(u) + λk,c − Cu , (3.12)
where f (u) = (1/2)uHu + hu + r as defined in Section 3.1. We then have that the Lagrangian of (3.3) evaluated atλkisL(u, λk) = f(u). The function f(u) is strongly convex with constant μ≥ λmin(R) > 0, where λmin(R) denotes the smallest eigenvalue of R. Strong
convexity of f(u) with modulus μ follows directly. Using (3.12), it holds that
f(u) − f(uk)≥ μ 2u − u k2 Hu, ∀u ∈ Hu , or, equivalently, L(u, λk)− L(uk,λk)≥ μ 2u − u k2 Hu, ∀u ∈ Hu . (3.13)
Substituting u = u∞ in (3.13) and by observing that max λ≤0 L(u ∞,λ) ≥ L(u∞,λk), we have that L(u∞,λ∞)− L(uk,λk)≥ μ 2u ∞− uk2 Hu, ∀u ∈ Hu . (3.14)
We have managed to derive an upper bound for the distance of the generated sequence of primal minimizers{uk} from the optimal one. The last step is to show that the Lagrangian
3.6 Convergence results 37
L(u, λ) is associated to the composite objective F (λ). This can be easily shown as follows: L(uk,λk) = min u∈Hu f (u) + λk,c − Cu =− max u∈Hu −f(u) + λk,Cu+λk,c =−f(Cλk) +λk,c =−F (λk), by (3.6) .
From strong duality and the fact that −F (λk) converges to the optimal dual value (first point), we have that the optimal value of the dual function −F (λ) coin- cides with that of the Lagrangian evaluated at the saddle point (u∞,λ∞), i.e.,
L(u∞,λ∞) = max
λ∈Hλ{−F (λ)} = −F (λ
∞) (see [BV04, Section 5.5.5]). Making use of the first point, inequality (3.14) becomes
μ 2u k− u∞2 Hu ≤ F (λ k)− F (λ∞)≤ a2Lλ0− λ∞2Hλ 2(k + a− 1)2 , (3.15)
which concludes the proof.
4. The state sequence is generated by
xk=Axinit+Buk . (3.16)
Strong convergence of the input sequence {uk}, along with the facts that B : Hx → Hu is bounded (follows directly from Lemma 7 in Appendix 3.10.2) and the uniqueness ofu∞prove strong convergence of the state sequence with rate 1/k, i.e.,
xk− x∞H x=B(uk− u∞)Hx ≤ Buk− u∞Hx ≤ aB & L μ λ0− λ∞H λ (k + a− 1) , with the last inequality following directly from the third point.
5. The proof for the unaccelerated case was presented in [SKJ14]. We give below the complete version of the proof taking into account the appearance of the over-relaxed sequences {ˆxk},
{ˆuk}, which renders the derivation of the result slightly more challenging.
The key piece for the proof is the weak convergence of ˆxk tox∞. This claim is subsequently proven as a sequence of intermediate results. We show that:
(a) The relaxed dual sequence{ˆλk} converges weakly to a dual minimizer λ∞.
(b) Provided that the operator C is bounded, the sequence {H ˆuk} converges weakly to
primal optimizeru∞.
(c) Weak convergence of the accelerated state sequence{ˆxk} to x∞ follows directly. Weak convergence of the relaxed sequence {ˆλk} follows from Corollary 2 of [CD15], which states that the error sequence{λk− λk−12} converges to zero with rate 1/k2. We state the result below.
Lemma 4. The relaxed sequence{ˆλk} converges weakly to λ∞.
Proof: Sinceλk− λk−12→ 0 and αk is bounded we also haveνk =√αk(λk− λk−1)→ 0. Since strong convergence implies weak convergence we have thatνk,y−−−→ 0, ∀y ∈ Hk→∞ λ. The relaxed sequence of duals ˆλ can be written as ˆλk = λk+√αkνk. Consequently, since λk λ∞, we have that
ˆλk,y = λk+√αkνk,y
=λk,y +√αkνk,y → λ∞
for all y∈ Hλ and hence ˆλk λ∞.
Lemma 5. The sequence{ˆuk} converges weakly to u∞.
Proof: Writing down the relation between ˆu and ˆλ from Lemma 2 in terms of the operators, we get ˆuk = H−1(CW ˆλk − h). Similarly we have uk = H−1(CW λk − h). Now, from Theorem 5 we haveuk→ u∞and from Lemma 4 we have ˆλk− λk 0. Therefore, sinceC, W and H−1 are bounded operators (see Lemmas 7 and 8 in Appendix 3.10.2) and since weak convergence is preserved under bounded linear mappings, we conclude that ˆuku∞. Lemma 6. The sequence{ˆxk} converges weakly to x∞.
Proof: Exactly as we did at the fourth point of Theorem 5, the accelerated state sequence can be written as
ˆ
xk=Ax
init+B ˆuk .
Weak convergence ˆuk u∞and boundedness ofB prove weak convergence of the accelerated
state sequence to x∞.
We have, hence, proven the weak convergence of the accelerated state sequence to the optimal one. For{Tk} to be bounded, it is sufficient to show that
lim sup
k→∞ T
k<∞. (3.17)
To prove (3.17), define the sequence of the first hitting times of the interior ofS as