22.1
The given function is a quadratic, which we represent in the form
f = x> −1 −α 1 −α −1 −2 1 −2 −5 x.
A quadratic function is concave if and only if it is negative-semidefinite. Equivalently, if and only if its negative is positive-semidefinite. On the other hand, a symmetric matrix is positive semidefinite if and only
if all its principal minors, not just the leading principal minors, are nonnegative. Thus we will determine the range of the parameter α for which
−f = x> 1 α −1 α 1 2 −1 2 5 x.
is positive-semidefinite. It is easy to see that the three first-order principal minors (diagonal elements of F ) are all positive. There are three second-order principal minors. Only one of them, the leading principal minor, is a function of the parameter α,
det F (1 : !2, 1 : !2) = det " 1 α α 1 # = 1 − α2.
The above second-order leading principal minor is nonnegative if and only if α ∈ [−1, 1].
The other second-order principal minors are
det F (1 : !3, 1 : !3) and det F (2 : !3, 2 : !3), and they are positive. There is only one third-order principal minor, det F , where
det F = det " 1 2 2 5 # − α det " α −1 2 5 # − det " α −1 1 2 # = 1 − α(5α + 2) − (2α + 1) = 1 − 5α2− 2α − 2α − 1 = −5α2− 4α.
The third-order principal minor is nonnegative if and only if, −α(5α + 4) ≥ 0, that is, if and only if α ∈ [−4/5, 0].
Combining this with α ∈ [−1, 1] from above, we conclude that the function f is negative-semidefinite, equivalently, the quadratic function f is concave, if and only if
α ∈ [−4/5, 0]. 22.2 We have φ(α) = 1 2(x + αd) >Q(x + αd) − (x + αd)b = 1 2(d > Qd)α2+ d>(Qx − b)α + 1 2x >Qx + x>b . This is a quadratic function of α. Since Q > 0, then
d2φ
dα2(α) = d
>Qd > 0
and hence by Theorem 22.5, φ is strictly convex. 22.3 Write f (x) = x>Qx, where Q = 1 2 " 0 1 1 0 # .
Let x, y ∈ Ω. Then, x = [a1, ma1]> and y = [a2, ma2]> for some a1, a2 ∈ R. By Proposition 22.1, it is
enough to show that (y − x)>Q(y − x) ≥ 0. By substitution,
(y − x)>Q(y − x) = m(a2− a1)2≥ 0,
which completes the proof. 22.4
Let x, y ∈ Ω and α ∈ (0, 1). Then, h(x) = h(y) = c. By convexity of Ω, h(αx + (1 − α)y) = c. Therefore, h(αx + (1 − α)y) = αh(x) + (1 − α)h(y)
and so h is convex over Ω. We also have
−h(αx + (1 − α)y) = α(−h(x)) + (1 − α)(−h(y)), which shows that −h is convex, and thus h is concave.
22.5
At x = 0, for ξ ∈ [−1, 1], we have, for all y ∈ R,
|y| ≥ |0| + ξ(y − 0) = ξy.
Thus, in this case any ξ in the interval [−1, 1] is a subgradient of f at x = 0. At x = 1, ξ = 1 is the only subgradient of f , because, for all y ∈ R,
|y| ≥ 1 + ξ(y − 1) = y. 22.6
Let x, y ∈ Ω and α ∈ (0, 1). For convenience, write ¯f = max{f1, . . . , f`} = maxifi. We have
¯
f (αx + (1 − α)y) = max
i fi(αx + (1 − α)y)
≤ max
i (αfi(x) + (1 − α)fi(y)) by convexity of each fi
≤ α max
i fi(x) + (1 − α) maxi fi(y) by property of max
= α ¯f (x) + (1 − α) ¯f (y) which implies that ¯f is convex.
22.7
⇒: This is true by definition.
⇐: Let d ∈ Rn be given. We want to show that d>Qd ≥ 0. Now, fix some vector x ∈ Ω. Since Ω is
open, there exists α 6= 0 such that y = x − αd ∈ Ω. By assumption, 0 ≤ (y − x)>Q(y − x) = α2d>Qd which implies that d>Qd ≥ 0.
22.8
Yes, the problem is a convex optimization problem.
First we show that the objective function f (x) =12kAx − bk2 is convex. We write
f (x) = 1 2x
>(A>A)x − (b>A)x + constant
which is a quadratic function with Hessian A>A. Since the Hessian A>A is positive semidefinite, the objective function f is convex.
Next we show that the constraint set is convex. Consider two feasible points x and y, and let λ ∈ (0, 1). Then, x and y satisfy e>x = 1, x ≥ 0 and e>y = 1, y ≥ 0, respectively. We have
Moreover, each component of λx + (1 − λ)y is given by λxi+ (1 − λ)yi, which is nonnegative because every
term here is nonnegative. Hence, λx + (1 − λ)y is a feasible point, which shows that the constraint set is convex.
22.9
We need to show that Ω is a convex set, and f is a convex function on Ω.
To show that Ω is a convex set, we need to show that for any y, z ∈ Ω and α ∈ (0, 1), we have αy+(1−α)z ∈ Ω. Let y, z ∈ Ω and α ∈ (0, 1). Thus, y1= y2≥ 0 and z1= z2≥ 0. Hence,
x = αy + (1 − α)z = " αy1+ (1 − α)z1 αy2+ (1 − α)z2 # Now, x1= αy1+ (1 − α)z1= αy2+ (1 − α)z2= x2, and since α, 1 − α ≥ 0, x1≥ 0.
Hence, x ∈ Ω and therefore Ω is convex.
To show that f is convex on Ω, we need to show that for any y, z ∈ Ω and α ∈ [0, 1], f (αy + (1 − α)z) ≤ αf (y) + (1 − α)f (z). Let y, z ∈ Ω and α ∈ [0, 1]. Thus, y1= y2≥ 0 and z1= z2≥ 0, so that f (y) = y31and
f (z) = z3
1. Also, α3≤ α and (1 − α)3≤ (1 − α). We have
f (αy + (1 − α)z) = (αy1+ (1 − α)z1)3 = α3y13+ (1 − α)3z13+ 3α2x21(1 − α)y1+ 3αx1(1 − α)2y21 ≤ αy31+ (1 − α)z13+ max(y1, z1)(α3− α + (1 − α)3− (1 − α) + 3α2(1 − α) + 3α(1 − α)2) = αy31+ (1 − α)z13 = αf (y) + (1 − α)f (z). Hence, f is convex. 22.10
Since the problem is a convex optimization problem, we know for sure that any point of the form αy+(1−α)z, α ∈ (0, 1), is a global minimizer. However, any other point may or may not be a minimizer. Hence, the largest set of points G ⊂ Ω for which we can be sure that every point in G is a global minimizer, is given by
G = {αy + (1 − α)z : 0 ≤ α ≤ 1}. 22.11
a. Let f be the objective function and Ω the constraint set. Consider the set Γ = {x ∈ Ω : f (x) ≤ 1}. This set contains all three of the given points. Moreover, by Lemma 22.1, Γ is convex. Now, if we take the average of the first two points (which is a convex combination of them), the resulting point (1/2)[1, 0, 0]>+ (1/2)[0, 1, 0]> = (1/2)[1, 1, 0]> is in Γ, because Γ is convex. Similarly, the point (2/3)(1/2)[1, 1, 0]> + (1/3)[0, 0, 1]> = (1/3)[1, 1, 1]> is also in Γ, because Γ is convex. Hence, the objective function value of (1/3)[1, 1, 1]> must be ≤ 1.
b. If the three points are all global minimizers, then the point (1/3)[1, 1, 1]>, which must cannot have higher objective function value than the given three points (by part a), must also be a global minimizer.
22.12
a. The Lagrange condition for the problem is given by:
x>Q + λ>A = 0 Ax = b. From the first equation above, we obtain
Applying the second equation (constraint on x), we have (AQ−1A>)λ = b.
Since rank A = m, the matrix AQ−1A>is invertible. Therefore, the only solution to the Lagrange condition is
x = Q−1A>(AQ−1A>)−1b.
b. The point in part a above is a global minimizer because the problem is a convex optimization problem (by problem 1, the constraint set is convex; the objective function is convex because its Hessian, Q, is positive definite).
22.13
By Theorem 22.4, for all x ∈ Ω, we have
f (x) ≥ f (x∗) + Df (x∗)(x − x∗). Substituting Df (x∗) from the equation Df (x∗) +P
j∈J (x∗)µ∗ja>j = 0> into the above inequality yields
f (x) ≥ f (x∗) − X
j∈J (x∗)
µ∗ja>j(x − x∗).
Observe that for each j ∈ J (x∗),
a>jx∗+ bj = 0,
and for each x ∈ Ω,
a>jx + bj≥ 0.
Hence, for each j ∈ J (x∗),
a>j(x − x∗) ≥ 0. Since µ∗j ≤ 0, we get
f (x) ≥ f (x∗) − X
j∈J (x∗)
µ∗ja>j(x − x∗) ≥ f (x∗)
and the proof is completed. 22.14
a. Let Ω = {x ∈ Rn: a>x ≥ b}, x1, x2∈ Ω, and λ ∈ [0, 1]. Then, a>x1≥ b and a>x2≥ b. Therefore,
a>(λx1+ (1 − λ)x2) = λa>x1+ (1 − λ)a>x2
≥ λb + (1 − λ)b = b
which means that λx1+ (1 − λ)x2∈ Ω. Hence, Ω is a convex set.
b. Rewrite the problem as
minimize f (x) subject to g(x) ≤ 0
where f (x) = kxk2 and g(x) = b − a>x. Now, ∇g(x) = −a 6= 0. Therefore, any feasible point is regular.
By the Karush-Kuhn-Tucker theorem, there exists µ∗≥ 0 such that 2x∗− µ∗a = 0
µ∗(b − a>x∗) = 0.
Since x∗ is a feasible point, then x∗ 6= 0. Therefore, by the first equation, we see that µ∗6= 0. The second