21.1
a. We form the Lagrangian function,
l(x, µ) = x21+ 4x22+ µ(4 − x21− 2x22).
The KKT conditions take the form,
Dxl(x, µ) = h 2x1− 2µx1 8x2− 4µx2 i = 0> µ(4 − x21− 2x2 2) = 0 µ ≥ 0 4 − x21− 2x2 2≤ 0.
From the first of the above equality, we obtain
(1 − µ)x1= 0 (2 − µ)x2= 0.
We first consider the case when µ = 0. Then, we obtain the point, x(1) = 0, which does not satisfy the constraints.
The next case is when µ = 1. Then we have to have x2= 0 and using µ(4 − x21− 2x22) = 0 gives
x(2)= " 2 0 # and x(3)= − " 2 0 # .
For the case when µ = 2, we have to have x1= 0 and we get
x(4)= " 0 √ 2 # and x(5)= − " 0 √ 2 # . b. The Hessian of l is L = " 2 0 0 8 # + µ " −2 0 0 −4 # . When µ = 1, L = " 0 0 0 4 # . We next find the subspace
˜
T = T = {y :h±4 0iy = 0} = {y = ah0 1i
>
: a ∈ R}. We then check for positive definiteness of L on ˜T ,
y>Ly = a2h0 1i " 0 0 0 4 # " 0 1 # = 4a2> 0.
Hence, x(2) and x(3) satisfy the SOSC to be strict local minimizers.
When µ = 2, L = " −2 0 0 0 # , and T = {y = ah1 0i > : a ∈ R}.
We have
y>Ly = −2a2< 0. Thus, x(4) and x(5) do not satisfy the SONC to be minimizers.
In summary, only x(2) and x(3) are strict local minimizers.
21.2
a. We first find critical points by applying the Karush-Kuhn-Tucker conditions, which are 2x1− 2 − 2µ1x1+ 5µ2 = 0 2x2− 10 + 1 5µ1+ 1 2µ2 = 0 µ1 1 5x2− x 2 1 + µ2 5x1+ 1 2x2− 5 = 0 µ ≥ 0. We have to check four possible combinations.
Case 1: (µ1= 0, µ2= 0) Solving the first and second Karush-Kuhn-Tucker equations yields x(1) = [1, 5]>.
However, this point is not feasible and is therefore not a candidate minimizer. Case 2: (µ1> 0, µ2= 0) We have two possible solutions:
x(2)= [−0.98, 4.8]> µ(2)1 = 2.02 x(3)= [−0.02, 0]> µ(3)1 = 50.
Both x(2) and x(3) satisfy the constraints, and are therefore candidate minimizers.
Case 3: (µ1= 0, µ2> 0) Solving the corresponding equation yields:
x(4)= [0.5050, 4.9505]> µ(4)1 = 0.198. The point x(4) is not feasible, and hence is not a candidate minimizer.
Case 4: (µ1> 0, µ2> 0) We have two solutions:
x(5)= [0.732, 2.679]> µ(5)= [13.246, 3.986]> x(6) = [−2.73, 37.32]> µ(6)= [188.8, −204]>. The point x(5) is feasible, but x(6) is not.
We are left with three candidate minimizers: x(2), x(3), and x(5). It is easy to check that they are regular.
We now check if each satisfies the second order conditions. For this, we compute L(x, µ) = " 2 − 2µ1 0 0 2 # . For x(2), we have L(x(2), µ(2)) = " −2.04 0 0 2 # ˜ T (x(2)) = {a[−0.1021, 1]> : a ∈ R}. Let y = a[−0.1021, 1]>∈ ˜T (x(2)) with a 6= 0. Then
y>L(x(2), µ(2))y = 1.979a2> 0. Thus, by the SOSC, x(2) is a strict local minimizer.
For x(3), we have L(x(3), µ(3)) = " −97.958 0 0 2 # ˜ T (x(3)) = {a[−4.898, 1]>: a ∈ R}.
Let y = a[−4.898, 1]>∈ ˜T (x(3)) with a 6= 0. Then,
y>L(x(3), µ(3))y = −2347.9a2< 0.
Thus, x(3) does not satisfy the SOSC. In fact, in this case, we have T (x(3)) = ˜T (x(3)), and hence x(3) does not satisfy the SONC either. We conclude that x(3) is not a local minimizer. We can easily check that x(3) is not a local maximizer either.
For x(5), we have L(x(5), µ(5)) = " −24.4919 0 0 2 # ˜ T (x(5)) = {0}.
The SOSC is trivially satisfied, and therefore x(5) is a strict local minimizer.
b. The Karush-Kuhn-Tucker conditions are:
2x1− µ1− µ3 = 0 2x2− µ2− µ3 = 0 −x1 ≤ 0 −x2 ≤ 0 −x1− x2+ 5 ≤ 0 −µ1x1− µ2x2+ µ3(−x1− x2+ 5) = 0 µ1, µ2, µ3 ≥ 0.
It is easy to verify that the only combination of Karush-Kuhn-Tucker multipliers resulting in a feasible point is µ1= µ2= 0, µ3> 0. For this case, we obtain x∗= [2.5, 2.5]>, µ∗= [0, 0, 5]>. We have
L(x∗, µ∗) = " 2 0 0 2 # > 0.
Hence, x∗ is a strict local minimizer (in fact, the only one for this problem). c. The Karush-Kuhn-Tucker conditions are:
2x1+ 6x2− 4 + 2µ1x1+ 2µ2 = 0 6x1− 2 + 2µ1− 2µ2 = 0 x21+ 2x2− 1 ≤ 0 2x1− 2x2− 1 ≤ 0 µ1(x21+ 2x2− 1) + µ2(2x1− 2x2− 1) = 0 µ1, µ2 ≥ 0.
It is easy to verify that the only combination of Karush-Kuhn-Tucker multipliers resulting in a feasible point is µ1= 0, µ2> 0. For this case, we obtain x∗= [9/14, 2/14]>, µ∗= [0, 13/14]>. We have
L(x∗, µ∗) = " 2 6 6 0 # ˜ T (x∗) = {a[1, 1] : a ∈ R}. Let y = a[1, 1] ∈ ˜T (x∗) with a 6= 0. Then
y>L(x∗, µ∗)y = 14a2> 0.
21.3
The Karush-Kuhn-Tucker conditions are:
2x1+ 2λx1+ 2λx2+ 2µx1 = 0 2x2+ 2λx1+ 2λx2− µ = 0 x21+ 2x1x2+ x22− 1 = 0 x21− x2 ≤ 0 µ(x21− x2) = 0 µ ≥ 0. We have two cases to consider.
Case 1: (µ > 0) Substituting x2= x21into the third equation and combining the result with the first two
yields two possible points:
x(1)= [−1.618, 0.618]> µ(1)= −3.7889 x(2)= [2.618, 0.382]> µ(2)= −0.2111.
Note that the resulting µ values violate the condition µ > 0. Hence, neither of the points are minimizers (although they are candidates for maximizers).
Case 2: (µ = 0) Subtracting the second equation from the first yields x1= x2, which upon substituting
into the third equation gives two possible points:
x(3)= [1/2, 1/2]>, x(4)= [−1/2, 1/2]>. Note that x(4) is not a feasible point, and is therefore not a candidate minimizer.
Therefore, the only remaining candidate is x(3), with corresponding λ(3) = −1/2 (and µ = 0). We now
check second order conditions. We have
L(x(3), 0, λ(3)) = " 1 −1 −1 1 # ˜ T (x(3)) = {a[1, −1]> : a ∈ R}. Let y = a[1, −1]>∈ ˜T (x(3)) with a 6= 0. Then
y>L(x(3), 0, λ(3))y = 4a2> 0. Therefore, by the SOSC, x(3) is a strict local minimizer.
21.4
The optimization problem is:
minimize e>np subject to Gp ≥ P em
p ≥ 0,
where G = [gi,j], en= [1, . . . , 1]> (with n components), and p = [p1, . . . , pn]>. The KKT condition for this
problem is:
e>n − µ>1G − µ>2 = 0>
µ>1(P em− Gp) − µ>2p = 0
Gp ≥ P em
21.5
a. We have f (x) = x2− (x1 − 2)3+ 3 and g(x) = 1 − x2. Hence, ∇f (x) = [−3(x1− 2)2, 1]> and
∇g(x) = [0, −1]>. The KKT condition is µ ≥ 0 −3(x1− 2)2 = 0 1 − µ = 0 µ(1 − x2) = 0 1 − x2 ≤ 0.
The only solution to the above conditions is x∗= [2, 1]>, µ∗= 1.
To check if x∗ is regular, we note that the constraint is active. We have ∇g(x∗) = [0, −1]>, which is nonzero. Hence, x∗ is regular.
b. We have L(x∗, µ∗) = F (x∗) + µ∗G(x∗) = " 0 0 0 0 # . Hence, the point x∗ satisfies the SONC.
c. Since µ∗ > 0, we have ˜T (x∗, µ∗) = T (x∗) = {y : [0, −1]y = 0} = {y : y2 = 0}, which means that ˜T
contains nonzero vectors. Hence, the SOSC does not hold at x∗. 21.6
a. Write f (x) = x2, g(x) = −(x2+ (x1− 1)2− 3). We have ∇f (x) = [0, 1]> and ∇g(x) = [−2(x1− 1), −1]>.
The KKT conditions are:
µ ≥ 0 −2µ(x1− 1) = 0
1 − µ = 0 µ(x2+ (x1− 1)2− 3) = 0
x2+ (x1− 1)2+ 3 ≤ 0.
From the third equation we get µ = 1. The second equation then gives x1= 1, and the fourth equation gives
x2= 3. Therefore, the only point that satisfies the KKT condition is x∗= [1, 3]>, with a KKT multiplier of
µ∗= 1.
b. Note that the constraint x2+ (x1− 1)2+ 3 ≥ 0 is active at x∗. We have T (x∗) = {y : [0, −1]y =
0} = {y : y2 = 0}, and N (x∗) = {y : y = [0, −1]z, z ∈ R} = {y : y1 = 0}. Because µ∗ > 0, we have
˜ T (x∗) = T (x∗) = {y : y2= 0}. c. We have L(x∗, µ∗) = O + 1 " −2 0 0 0 # = " −2 0 0 0 # .
From part b, T (x∗) = {y : y2= 0}. Therefore, for any y ∈ T (x∗), y>L(x∗, µ∗)y = −2y12≤ 0, which means
that x∗ does not satisfy the SONC. 21.7
a. We need to consider two optimization problems. We first consider the minimization problem minimize (x1− 2)2+ (x2− 1)2
subject to x21− x2≤ 0
x1+ x2− 2 ≤ 0
Then, we form the Lagrangian function
l(x, µ) = (x1− 2)2+ (x2− 1)2+ µ1(x21− x2) + µ2(x1+ x2− 2) + µ3(−x1).
The KKT condition takes the form ∇xl(x, µ) = h 2(x1− 2) + 2µ1x1+ µ2− µ3 2(x2− 1) − µ1+ µ2 i = 0T µ1(x21− x2) = 0 µ2(x1+ x2− 2) = 0 µ3(−x1) = 0 µi≥ 0.
The point x∗= 0 satisfies the above conditions for µ
1= −2, µ2= 0, and µ3= −4. Thus the point x∗ does
not satisfy the KKT conditions for minimum. We next consider the maximization problem
minimize − (x1− 2)2− (x2− 1)2
subject to x21− x2≤ 0
x1+ x2− 2 ≤ 0
− x1≤ 0.
The Lagrangian function for the above problem is,
l(x, µ) = −(x1− 2)2− (x2− 1)2+ µ1(x21− x2) + µ2(x1+ x2− 2) + µ3(−x1).
The KKT condition takes the form ∇xl(x, µ) = h −2(x1− 2) + 2µ1x1+ µ2− µ3 −2(x2− 1) − µ1+ µ2 i = 0> µ1(x21− x2) = 0 µ2(x1+ x2− 2) = 0 µ3(−x1) = 0 µi≥ 0.
The point x∗= 0 satisfies the above conditions for µ
1= 2, µ2= 0, and µ3= 4. Hence, the point x∗ satisfies
the KKT conditions for maximum.
b. We next compute the Hessian, with respect to x, of the lagrangian to obtain
L = F + µ∗1G1= " −2 0 0 −2 # + " 4 0 0 0 # = " 2 0 0 −2 #
which is indefinite on R2. We next find the subspace
˜ T = ( y : " ∇g1(x∗)> ∇g3(x∗)> # y = 0 ) = ( y : " 0 −1 −1 0 # y = 0 ) = {0},
That is, ˜T is a trivial subspace that consists only of the zero vector. Thus the SOSC for x∗ to be a strict local maximizer is trivially satisfied.
21.8
Note that all feasible points are regular. The KKT condition is: x22+ λ − µ = 0 2x1x2− λ = 0 µx1 = 0 µ ≥ 0 x1− x2 = 0 x1 ≥ 0.
We first try x1= x∗1= 0 (active inequality constraint). Substituting and manipulating, we have the solution
x∗
1 = x∗2 = 0 with µ∗ = 0, which is a legitimate solution. If we then try x1 = x∗1 > 0 (inactive inequality
constraint), we find that there is no consistent solution to the KKT condition. Thus, there is only one point satisfying the KKT condition: x∗= 0.
c. The tangent space at x∗= 0 is given by
T (0) = {y : [1, −1]y = 0, [−1, 0]y = 0} = {0}. Therefore, the SONC holds for the solution in part a.
d. We have L(x, λ, µ) = " 0 2x2 2x2 2x1 # .
Hence, at x∗= 0, we have L(0, 0, 0) = O. Since the active constraint at x∗= 0 is degenerate, we have ˜
T (0, 0) = {y : [1, −1]y = 0},
which is nontrivial. Hence, for any nonzero vector y ∈ ˜T (0, 0), we have y>L(0, 0, 0)y = 0 6> 0. Thus, the SOSC does not hold for the solution in part a.
21.9
a. The KKT condition for the problem is:
(Ax − b)>A + λe>− µ> = 0> µ>x = 0 µ ≥ 0 e>x − 1 = 0 x ≥ 0 where e = [1, . . . , 1]>.
b. A feasible point x∗ is regular in this problem if the vectors e, ei, i ∈ J (x∗) are linearly independent,
where J (x∗) = {i : x∗i = 0} and ei is the vector with 0 in all components except the ith component, which
is 1.
In this problem, all feasible points are regular. To see this, note that 0 is not feasible. Therefore, any feasible point results in the set J (x∗) having fewer than n elements, which implies that the vectors e, ei,
i ∈ J (x∗) are linearly independent. 21.10
By the KKT Theorem, there exists µ∗≥ 0 such that
(x∗− x0) + µ∗∇g(x∗) = 0
µ∗g(x∗) = 0. Premultiplying both sides of the first equation by (x∗− x0)>, we obtain
Since kx∗− x0k2> 0 (because g(x0) > 0) and µ∗≥ 0, we deduce that (x∗− x0)>∇g(x∗) < 0 and µ∗ > 0.
From the second KKT condition above, we conclude that g(x∗) = 0. 21.11
a. By inspection, we guess the point [2, 2]> (drawing a picture may help).
b. We write f (x) = (x1− 3)2+ (x2− 4)2, g1(x) = −x1, g2(x) = −x2, g3(x) = x1− 2, g4(x) = x2− 2,
g = [g1, g2, g3, g4]>. The problem becomes
minimize f (x) subject to g(x) ≤ 0.
We now check the SOSC for the point x∗ = [2, 2]>. We have two active constraints: g3, g4. Regularity
holds, since ∇g3(x∗) = [1, 0]> and ∇g4(x∗) = [0, 1]>. We have ∇f (x∗) = [−2, −4]>. We need to find a
µ∗∈ R4, µ∗≥ 0, satisfying FONC. From the condition µ∗>g(x∗) = 0, we deduce that µ∗
1= µ∗2= 0. Hence,
Df (x∗) + µ∗>Dg(x∗) = 0> if and only if µ∗= [0, 0, 2, 4]>. Now, F (x∗) = " 2 0 0 2 # , [µ∗G(x∗)] = " 0 0 0 0 # . Hence L(x∗, µ∗) = " 2 0 0 2 #
which is positive definite on R2. Hence, SOSC is satisfied, and x∗ is a strict local minimizer.
21.12 The KKT condition is x>Q + µ>A = 0> µ>(Ax − b) = 0 µ ≥ 0 Ax − b ≤ 0. Postmultiplying the first equation by x gives
x>Qx + µ>Ax = 0. We note from the second equation that µ>Ax = µ>b. Hence,
x>Qx + µ>b = 0.
Since Q > 0, the first term is nonnegative. Also, the second term is nonnegative because µ ≥ 0 and b ≥ 0. Hence, we conclude that both terms must be zero. Because Q > 0, we must have x = 0.
Aside: Actually, we can deduce that the only solution to the KKT condition must be 0, as follows. The problem is convex; thus, the only points satisfying the KKT condition are global minimizers. However, we see that 0 is a feasible point, and is the only point for which the objective function value is 0. Further, the objective function is bounded below by 0. Hence, 0 is the only global minimizer.
21.13
a. We have one scalar equality constraint with h(x) = [c, d]>x − e and two scalar inequality constraints with g(x) = −x. Hence, there exists µ∗∈ R2and λ∗∈ R such that
µ∗ ≥ 0 a + cλ∗− µ∗1 = 0 b + dλ∗− µ∗2 = 0 µ∗>x∗ = 0 cx∗1+ dx∗2 = e x∗ ≥ 0.
b. Because x∗is a basic feasible solution, and the equality constraint precludes the point 0, exactly one of the inequality constraints is active. The vectors ∇h(x∗) = [c, d]> and ∇g1 = [1, 0]> are linearly independent.
Similarly, the vectors ∇h(x∗) = [c, d]> and ∇g2 = [0, 1]> are linearly independent. Hence, x∗ must be
regular.
c. The tangent space is given by
T (x∗) = {y ∈ Rn: Dh(x∗)y = 0, Dg
j(x∗)y = 0, j ∈ J (x∗)}
= N (M ),
where M is a matrix with the first row equal to Dh(x∗) = [c, d], and the second row is either Dg1= [1, 0]
or Dg2= [0, 1]. But, as we have seen in part b, rank M = 2 Hence, T (x∗) = {0}.
d. Recall that we can take µ∗to be the relative cost coefficient vector (i.e., the KKT conditions are satisfied with µ∗ being the relative cost coefficient vector). If the relative cost coefficients of all nonbasic variables are strictly positive, then µ∗j > 0 for all j ∈ J (x∗). Hence, ˜T (x∗, µ∗) = T (x∗) = {0}, which implies that L(x∗, λ∗, µ∗) > 0 on ˜T (x∗, µ∗). Hence, the SOSC is satisfied.
21.14
Let x∗ be a solution. Since A is of full rank, x∗ is regular. The KKT Theorem states that x∗ satisfies: µ∗ ≥ 0
c>+ µ∗>A = 0 µ∗>Ax∗ = 0.
If we postmultiply the second equation by x∗ and subtract the third from the result, we get c>x∗= 0.
21.15
a. We can write the LP as
minimize f (x) subject to h(x) = 0,
g(x) ≤ 0,
where f (x) = c>x, h(x) = Ax − b, and g(x) = −x. Thus, we have Df (x) = c>, Dh(x) = A, and
Dg(x) = −I. The Karush-Kuhn-Tucker conditions for the above problem have the form: if x∗ is a local minimizer, then there exists λ∗ and µ∗such that
µ∗ ≥ 0 c>+ λ∗>A − µ∗> = 0>
µ∗>x∗ = 0.
b. Let x∗ be an optimal feasible solution. Then, x∗ satisfies the Karush-Kuhn-Tucker conditions listed in part a. Since µ∗≥ 0, then from the second condition in part a, we obtain (−λ∗)>A ≤ c>. Hence, ¯λ = −λ∗ is a feasible solution to the dual (see Chapter 17). Postmultiplying the second condition in part a by x∗, we have
0 = c>x∗+ λ∗>Ax − µ∗>x∗= c>x∗+ λ∗>b which gives
c>x∗= ¯λ>b.
Hence, ¯λ achieves the same objective function value for the dual as x∗ for the primal.
21.16
By definition of J (x∗), we have gi(x∗) < 0 for all i 6∈ J (x∗). Since by assumption gi is continuous for all i,
there exists ε > 0 such that gi(x) < 0 for all i 6∈ J (x∗) and all x in the set B = {x : kx − x∗k < ε}. Let
S1= {x : h(x) = 0, gj(x) ≤ 0, j ∈ J (x∗)}. We claim that S ∩ B = S1∩ B. To see this, note that clearly
S ∩ B ⊂ S1∩ B. To show that S1∩ B ⊂ S ∩ B, suppose x ∈ S1∩ B. Then, by definition of S1 and B, we
have h(x) = 0, gj(x) ≤ 0 for all j ∈ J (x∗), and gi(x) < 0 for all i 6∈ J (x∗). Hence, x ∈ S ∩ B.
Since x∗ is a local minimizer of f over S, and S ∩ B ⊂ S, x∗ is also a local minimizer of f over S ∩ B = S1∩ B. Hence, we conclude that x∗ is a regular local minimizer of f on S1. Note that S0 ⊂ S1,
and x∗ ∈ S0. Therefore, x∗ is a regular local minimizer of f on S0.
21.17
Write f (x) = x2
1+x22, g1(x) = x21−x2−4, g2(x) = x2−x1−2, and g = [g1, g2]>. We have ∇f (x) = [2x1, 2x2]>,
∇g1(x) = [2x1, −1]>, ∇g2(x) = [−1, 1]>, and D2f (x) = diag[2, 2]. We compute
∇f (x) + µ>∇g(x) = [2x1+ 2µ1x1− µ2, 2x2− µ1+ µ2]>.
We use the FONC to find critical points. Rewriting ∇f (x) + µ>∇g(x) = 0, we obtain x1= µ2 2 + 2µ1 , x2= µ1− µ2 2 . We also use µ>g(x) = 0 and µ ≥ 0, giving
µ1(x21− x2− 4) = 0, µ2(x2− x1− 2) = 0.
The vector µ has two components; therefore, we try four different cases. Case 1: (µ1> 0, µ2> 0) We have
x21− x2− 4 = 0, x2− x1− 2 = 0.
We obtain two solutions: x(1)= [−2, 0]>and x(2) = [3, 5]>. For x(1), the two FONC equations give µ1= µ2
and −2(2 + 2µ1) = µ1, which yield µ1= µ2= −4/5. This is not a legitimate solution since we require µ ≥ 0.
For x(2), the two FONC equations give µ
1− µ2= 10 and 3(2 + 2µ1) = µ2, which yield µ = [−16/5, −66/5].
Again, this is not a legitimate solution. Case 2: (µ1= 0, µ2> 0) We have x2− x1− 2 = 0, x1= µ2 2 , x2= − µ2 2 .
Hence, x1= −x2, and thus x = [−1, 1], µ2= −2. This is not a legitimate solution since we require µ ≥ 0.
Case 3: (µ1> 0, µ2= 0) We have
x21− x2− 4 = 0, x1= 0, x2=
µ1
2 . Therefore, x2= −4, µ1= −8, and again we don’t have a legitimate solution.
Case 4: (µ1 = 0, µ2 = 0) We have x1 = x2 = 0, and all constraints are inactive. This is a legitimate
candidate for the minimizer. We now apply the SOSC. Note that since the candidate is an interior point of the constraint set, the SOSC for the problem is equivalent to the SOSC for unconstrained optimization. The Hessian matrix D2f (x) = diag[2, 2] is symmetric and positive definite. Hence, by the SOSC, the point
x∗= [0, 0]> is the strict local minimizer (in fact, it is easy to see that it is a global minimizer). 21.18
Write f (x) = x2
1+x22, g1(x) = −x1+x22+4, g2(x) = x1−10, and g = [g1, g2]>. We have ∇f (x) = [2x1, 2x2]>,
∇g1(x) = [−1, 2x2]>, ∇g2(x) = [1, 0]>, D2f (x) = diag[2, 2], D2g1(x) = diag[0, 2], and D2g2(x) = O. We
compute
∇f (x) + µ>∇g(x) = [2x1− µ1+ µ2, 2x2+ 2µ1x2]>.
We use the FONC to find critical points. Rewriting ∇f (x) + µ>∇g(x) = 0, we obtain
x1=
µ1− µ2
Since we require µ ≥ 0, we deduce that x2= 0. Using µ>g(x) = 0 gives
µ1(−x1+ 4) = 0, µ2= 0.
We are left with two cases.
Case 1: (µ1> 0, µ2= 0) We have −x1+ 4 = 0, and µ1= 8, which is a legitimate candidate.
Case 2: (µ1 = 0, µ2 = 0) We have x1 = x2 = 0, which is not a legitimate candidate, since it is not a
feasible point.
We now apply SOSC to our candidate x = [4, 0]>, µ = [8, 0]>. Now,
L([4, 0]>, [8, 0]>) = " 2 0 0 2 # + 8 " 0 0 0 2 # = " 2 0 0 18 # ,
which is positive definite on all of R2. The point [4, 0]> is clearly regular. Hence, by the SOSC, x∗ = [4, 0] is a strict local minimizer.
21.19 Write f (x) = x2 1+x22, g1(x) = −x1−x22+4, g2(x) = 3x2−x1, g3(x) = −3x2−x1and g = [g1, g2, g3]>. We have ∇f (x) = [2x1, 2x2]>, ∇g1(x) = [−1, −2x2]>, ∇g2(x) = [−1, 3]>, ∇g2(x) = [−1, −3]>, D2f (x) = diag[2, 2], D2g 1(x) = diag[0, −2], and D2g2(x) = D2g3(x) = O.
From the figure, we see that the two candidates are x(1)= [3, 1] and x(2)= [3, −1]. Both points are easily
verified to be regular. For x(1), we have µ
3= 0. Now,
Df (x(1)) + µ>Dg(x(1)) = [6 − µ1− µ2, 2 − 2µ1+ 3µ2] = 0>,
which yields µ1 = 4, µ2 = 2. Now, ˜T (x(1)) = {0}. Therefore, any matrix is positive definite on ˜T (x(1)).
Hence, by the SOSC, x(1) is a strict local minimizer. For x(2), we have µ2= 0. Now,
Df (x(1)) + µ>Dg(x(1)) = [6 − µ1− µ3, −2 + 2µ1− 3µ3] = 0>,
which yields µ1= 4, µ3= 2. Now, again we have ˜T (x(2)) = {0}. Therefore, any matrix is positive definite
on ˜T (x(2)). Hence, by the SOSC, x(2) is a strict local minimizer.
21.20
a. Write f (x) = 3x1 and g(x) = 2 − x1− x22. We have ∇f (x∗) = [3, 0]> and ∇g(x∗) = [−1, 0]>. Hence,
letting µ∗= 3, we have ∇f (x∗)+µ∗∇g(x∗) = 0. Note also that µ∗≥ 0 and µ∗g(x∗) = 0. Hence, x∗= [2, 0]>
satisfies the KKT (first order necessary) condition.
b. We have F (x∗) = O and G(x∗) = diag[0, −2]. Hence, L(x∗, µ∗) = O + 3 diag[0, −2] = diag[0, −6]. Also, T (x∗) = {y : [−1, 0]y = 0} = {y : y1= 0}. Hence, x∗ = [2, 0]> does not satisfy the second order necessary
condition.
c. No. Consider points of the form x = [−x22+ 2, x2]>, x2 ∈ R. Such points are feasible, and could be
arbitrarily close to x∗. However, for such points x 6= x∗,
f (x) = 3(−x22+ 2) = 6 − 6x22< 6 = f (x∗).
Hence, x∗ is not a local minimizer.
21.21
The KKT condition for the problem is
µ∗ ≥ 0 x∗+ λ∗a − µ∗ = 0 µ∗>x∗ = 0.
Premultiplying the second KKT condition above by µ∗> and using the third condition, we get λ∗µ∗>a = kµ∗k2.
Also, premultiplying the second KKT condition above by x∗> and using the feasibility condition a>x∗= b, we get
λ∗= −kx
∗k2
b < 0.
We conclude that µ∗= 0. For if not, the equation λ∗µ∗>a = kµ∗k2implies that µ∗>a < 0, which contradicts
µ∗≥ 0 and a ≥ 0.
Rewriting the second KKT condition with µ∗= 0 yields
x∗= −λ∗a. Using the feasibility condition a>x∗= b, we get
x∗= a b kak2.
21.22
a. Suppose (x∗1)2+ (x∗
2)2 < 1. Then, the point x∗ = [x∗1, x∗2]> lies in the interior of the constraint set
Ω = {x : kxk2≤ 1}. Hence, by the FONC for unconstrained optimization, we have that ∇f (x∗) = 0, where
f (x) = kx − [a, b]>k2 is the objective function. Now, ∇f (x∗) = 2(x∗− [a, b]>) = 0, which implies that
x∗= [a, b]> which violates the assumption (x∗1)2+ (x∗ 2)2< 1.
b. First, we need to show that x∗ is a regular point. For this, note that if we write the constraint as g(x) = kxk2− 1 ≤ 0, then ∇g(x∗) = 2x∗ 6= 0. Therefore, x∗ is a regular point. Hence, by the Karush-
Kuhn-Tucker theorem, there exists µ ∈ R, µ ≥ 0, such that ∇f (x∗) + µ∇g(x∗) = 0, which gives x∗= 1 1 + µ " a b # .
Hence, x∗ is unique, and we can write x∗1= αa, x∗2= αb, where α = 1/(1 + µ) ≥ 0.
c. Using part b and the fact that kx∗k = 1, we get kx∗k2= α2k[a, b]>k2= 1, which gives α = 1/k[a, b]>k =
1/√a2+ b2.
21.23
a. The Karush-Kuhn-Tucker conditions for this problem are 2x∗1+ µ∗exp(x∗1) = 0
2(x∗2+ 1) − µ∗ = 0 µ∗(exp(x∗1) − x∗2) = 0 exp(x∗1) ≤ x∗2
µ∗ ≥ 0.
b. From the second equation in part a, we obtain µ∗ = 2(x∗2+ 1). Since x∗2 ≥ exp(x∗1) > 0, then µ∗ > 0.
Hence, by the third equation in part a, we obtain x∗
2= exp(x∗1).
c. Since µ∗= 2(x∗2+ 1) = 2(exp(x∗1) + 1), then by the first equation in part a, we have 2x∗1+ 2(exp(x∗1) + 1) exp(x∗1) = 0
which implies
Since exp(x∗1), exp(2x∗1) > 0, then x∗1< 0, and hence exp(x∗1), exp(2x∗1) < 1. Therefore, x∗1> −2. 21.24
a. We rewrite the problem as
minimize f (x) subject to g(x) ≤ 0,
where f (x) = c>x and g(x) = 12kxk2− 1. Hence, ∇f (x) = c and ∇g(x) = x. Note that x∗ 6= 0 (for
otherwise it would not be feasible), and therefore it is a regular point. By the KKT theorem, there exists µ∗ ≤ 0 such that c = µ∗x∗ and µ∗g(x∗) = 0. Since c 6= 0, we must have µ∗ 6= 0. Therefore, g(x∗) = 0,
which implies that kx∗k2= 2.
b. From part a, we have α2kek2= 2. Since kek2= n, we have α =p2/n.
To find c, we use
4 = c>x∗+ 8 = µ∗kx∗k2+ 8 = 2µ∗+ 8,
and thus µ∗= −2. Hence, c = −2αe = −2p2/ne. 21.25
We can represent the equivalent problem as
minimize f (x) subject to g(x) ≤ 0, where g(x) = 12kh(x)k2. Note that
∇g(x) = Dh(x)>h(x). Therefore, the KKT condition is:
µ∗ ≥ 0 ∇f (x∗) + µ∗Dh(x∗)>h(x∗) = 0
µ∗kh(x∗)k = 0.
Note that for a feasible point x∗, we have h(x∗) = 0. Therefore, the KKT condition becomes
µ∗ ≥ 0 ∇f (x∗) = 0.
Note that ∇g(x∗) = 0. Therefore, any feasible point x∗ is not regular. Hence, the KKT theorem cannot be applied in this case. This should be clear, since obviously ∇f (x∗) = 0 is not necessary for optimality in general.