3.1 Minimizing a function over the probability simplex. Find simple necessary and sufficient conditions for x∈ Rn to minimize a differentiable convex function f over the probability simplex,{x | 1Tx = 1, x 0}.
Solution. The simple basic optimality condition is that x is feasible, i.e., x 0, 1Tx = 1, and that∇f(x)T(y− x) ≥ 0 for all feasible y. We’ll first show this is equivalent to
i=1,...,nmin ∇f(x)i ≥ ∇f(x)Tx.
To see this, suppose that ∇f(x)T(y − x) ≥ 0 for all feasible y. Then in particular, for y = ei, we have ∇f(x)i ≥ ∇f(x)Tx, which is what we have above. To show the other way, suppose that
∇f(x)i ≥ ∇f(x)Tx holds, for i = 1, . . . , n. Let y be feasible, i.e., y 0, 1Ty = 1. Then multiplying
Now we can simplify even further. The condition above can be written as
i=1,...,nmin and it follows that
i=1,...,nmin
The right hand side is a mixture of ∂f /∂xi terms and equals the minimum of all of the terms. This is possible only if xk= 0 whenever ∂f /∂xk> mini∂f /∂xi.
Thus we can write the (necessary and sufficient) optimality condition as 1Tx = 1, x 0, and, for each k,
3.2 ‘Hello World’ in CVX*. Use CVX, CVX.PY or Convex.jl to verify the optimal values you obtained (analytically) for exercise 4.1 in Convex Optimization.
Solution.
(a) p? = 0.6
(b) p? =−∞
(c) p? = 0 (d) p? = 13 (e) p? = 12
%exercise 4.1 using CVX
%set up a vector to store optimal values of problems optimal_values=zeros(5,1);
%part a cvx_begin variable x(2)
minimize(x(1)+x(2)) 2*x(1)+x(2) >= 0;
x(1)+3*x(2) >= 1;
x >= 0;
cvx_end
optimal_values(1)=cvx_optval;
%part b cvx_begin variable x(2) minimize(-sum(x)) 2*x(1)+x(2) >= 0;
x(1)+3*x(2) >= 1;
x >= 0;
cvx_end
optimal_values(2)=cvx_optval;
%part c cvx_begin variable x(2) minimize(x(1)) 2*x(1)+x(2) >= 0;
x(1)+3*x(2) >= 1;
x >= 0;
cvx_end
optimal_values(3)=cvx_optval;
%part d cvx_begin variable x(2) minimize(max(x))
2*x(1)+x(2) >= 0;
x(1)+3*x(2) >= 1;
x >= 0;
cvx_end
optimal_values(4)=cvx_optval;
%part e cvx_begin
variable x(2,1)
minimize( square(x(1))+ 9*square(x(2)) ) 2*x(1)+x(2) >= 0;
x(1)+3*x(2) >= 1;
x >= 0;
cvx_end
optimal_values(5)=cvx_optval;
import cvxpy as cvx import numpy as np x1 = cvx.Variable() x2 = cvx.Variable()
constraints = [2*x1 + x2 >= 1, x1 + 3*x2 >= 1, x1 >= 0, x2 >= 0]
prob = cvx.Problem([], constraints)
# (a)
prob.objective = cvx.Minimize(x1+x2) prob.solve()
print ’With obj. x1+x2, p* is %.2f, optimal x1 is %.2f, optimal x2 is %.2f’ \
% (prob.value, x1.value, x2.value)
# (b)
prob.objective = cvx.Minimize(-x1-x2) prob.solve(solver=cvx.CVXOPT)
print ’With obj. -x1-x2, status is ’ + prob.status
# (c)
prob.objective = cvx.Minimize(x1) prob.solve()
print ’With obj. x1, p* is %.2f, x1* is %.2f, x2* is %.2f’ \
% (prob.value, x1.value, x2.value)
# (d)
prob.objective = cvx.Minimize(cvx.max_elemwise(x1, x2)) prob.solve()
print ’With obj. max(x1,x2), p* is %.2f, x1* is %.2f, x2* is %.2f’ \
# (e)
prob.objective = cvx.Minimize(cvx.square(x1) + 9*cvx.square(x2)) prob.solve()
print ’With obj. x1^2+9x2^2, p* is %.2f, x1* is %.2f, x2* is %.2f’ \
% (prob.value, x1.value, x2.value)
# exercise 4.1 using CVX using Convex
# set up a vector to store optimal values of problems optimal_values=zeros(5);
# part a
x = Variable(2) obj = x[1] + x[2]
constr = [2*x[1]+x[2] >= 1, x[1]+3*x[2] >= 1, x >= 0]
p = minimize(obj, constr) solve!(p)
optimal_values[1]=p.optval;
# part b
x = Variable(2) obj = -sum(x)
constr = [2*x[1]+x[2] >= 1, x[1]+3*x[2] >= 1, x >= 0]
p = minimize(obj, constr) solve!(p)
optimal_values[2]=p.optval;
# part c
x = Variable(2) obj = x[1]
constr = [2*x[1]+x[2] >= 1, x[1]+3*x[2] >= 1, x >= 0]
p = minimize(obj, constr) solve!(p)
optimal_values[3]=p.optval;
# part d
x = Variable(2) obj = maximum(x)
constr = [2*x[1]+x[2] >= 1, x[1]+3*x[2] >= 1, x >= 0]
p = minimize(obj, constr) solve!(p)
optimal_values[4]=p.optval;
# part e
x = Variable(2)
obj = square(x[1]) + 9*square(x[2])
constr = [2*x[1]+x[2] >= 1, x[1]+3*x[2] >= 1, x >= 0]
p = minimize(obj, constr) solve!(p)
optimal_values[5]=p.optval;
3.3 Reformulating constraints in CVX*. Each of the following CVX code fragments describes a convex constraint on the scalar variables x, y, and z, but violates the CVX rule set, and so is invalid.
Briefly explain why each fragment is invalid. Then, rewrite each one in an equivalent form that conforms to the CVX rule set. In your reformulations, you can use linear equality and inequality constraints, and inequalities constructed using CVX functions. You can also introduce additional variables, or use LMIs. Be sure to explain (briefly) why your reformulation is equivalent to the original constraint, if it is not obvious.
Check your reformulations by creating a small problem that includes these constraints, and solving it using CVX. Your test problem doesn’t have to be feasible; it’s enough to verify that CVX processes your constraints without error.
Remark. This looks like a problem about ‘how to use CVX software’, or ‘tricks for using CVX’.
But it really checks whether you understand the various composition rules, convex analysis, and constraint reformulation rules.
(a) norm([x + 2*y, x - y]) == 0 (b) square(square(x + y)) <= x - y
(c) 1/x + 1/y <= 1; x >= 0; y >= 0
(d) norm([max(x,1), max(y,2)]) <= 3*x + y (e) x*y >= 1; x >= 0; y >= 0
(f) (x + y)^2/sqrt(y) <= x - y + 5 (g) x^3 + y^3 <= 1; x >= 0; y >= 0
(h) x + z <= 1 + sqrt(x*y - z^2); x >= 0; y >= 0 Solution.
(a) The lefthand side is correctly identified as convex, but equality constraints are only valid with affine left and right hand sides. Since the norm of a vector is zero if and only if the vector is zero, we can express the constraint as x + 2*y == 0; x - y == 0, or simply x == 0; y == 0.
(b) The problem is that square() can only accept affine arguments, because it is convex, but not increasing. To correct this use square_pos() instead:
square_pos(square(x + y)) <= x - y
We can also reformulate this constraint by introducing an additional variable.
variable t
square(x + y) <= t
Note that, in general, decomposing the objective by introducing new variables doesn’t need to work. It works in this case because the outer square function is convex and monotonic over R+.
Alternatively, we can rewrite the constraint as (x + y)^4 <= x - y
(c) 1/x isn’t convex, unless you restrict the domain to R++. We can write this one as inv_pos(x) + inv_pos(y) <= 1.
The inv_pos function has domain R++ so the constraints x > 0, y > 0 are (implicitly) in-cluded.
(d) The problem is that norm() can only accept affine argument since it is convex but not increas-ing. One way to correct this is to introduce new variables u and v:
norm([u, v]) <= 3*x + y max(x,1) <= u
max(y,2) <= v
Decomposing the objective by introducing new variables works here because norm is convex and monotonic over R2+, and in particular over [1,∞) × [2, ∞).
(e) xy isn’t concave, so this isn’t going to work as stated. But we can express the constraint as x >= inv_pos(y). (You can switch around x and y here.) Another solution is to write the constraint as geo_mean([x, y]) >= 1. We can also give an LMI representation:
[x 1; 1 y] == semidefinite(2)
(f) This fails when we attempt to divide a convex function by a concave one. We can write this as
quad_over_lin(x + y, sqrt(y)) <= x - y + 5
This works because quad_over_lin is monotone decreasing in the second argument, so it can accept a concave function here, and sqrt is concave.
(g) The function x3+ y3 is convex for x ≥ 0, y ≥ 0. But x3 isn’t convex for x < 0, so CVX is going to reject this statement. One way to rewrite this constraint is
quad_pos_over_lin(square(x),x) + quad_pos_over_lin(square(y),y) <= 1
This works because quad_pos_over_lin is convex and increasing in its first argument, hence accepts a convex function in its first argument. (The function quad_over_lin, however, is not increasing in its first argument, and so won’t work.)
Alternatively, and more simply, we can rewrite the constraint as pow_pos(x,3) + pow_pos(y,3) <= 1
(h) The problem here is that xy isn’t concave, which causes CVX to reject the statement. To correct this, notice that
q
xy− z2 = q
y(x− z2/y), so we can reformulate the constraint as
x + z <= 1 + geo_mean([x - quad_over_lin(z,y), y])
This works, since geo_mean is concave and nondecreasing in each argument. It therefore accepts a concave function in its first argument.
We can check our reformulations by writing the following feasibility problem in CVX (which is obviously infeasible)
inv_pos(x) + inv_pos(y) <= 1;
norm([u; v]) <= 3*x + y;
pow_pos(x,3) + pow_pos(y,3) <= 1;
x+z <= 1+geo_mean([x-quad_over_lin(z,y), y]) cvx_end
3.4 Optimal activity levels. Solve the optimal activity level problem described in exercise 4.17 in Convex Optimization, for the instance with problem data
A =
You can do this by forming the LP you found in your solution of exercise 4.17, or more directly, using CVX*. Give the optimal activity levels, the revenue generated by each one, and the total revenue generated by the optimal solution. Also, give the average price per unit for each activity level, i.e., the ratio of the revenue associated with an activity, to the activity level. (These numbers should be between the basic and discounted prices for each activity.) Give a very brief story explaining, or at least commenting on, the solution you find.
Solution. For this part, we write the problem in a form close to its original statement, and let CVX* do the work of reformulating it as an LP.
The following CVX code implements this:
A=[ 1 2 0 1;
0 0 3 1;
0 3 1 1;
1 0 3 2];
cmax=[100;100;100;100;100];
p=[3;2;7;6];
pdisc=[2;1;4;2];
q=[4; 10 ;5; 10];
cvx_begin
variable x(4)
maximize( sum(min(p.*x,p.*q+pdisc.*(x-q))) ) subject to
x >= 0;
A*x <= cmax cvx_end
x
r=min(p.*x,p.*q+pdisc.*(x-q)) totr=sum(r)
avgPrice=r./x
import cvxpy as cvx import numpy as np
A = np.matrix(’1 2 0 1; \ 0 0 3 1; \ 0 3 1 1; \ 2 1 2 5; \ 1 0 3 2’)
cmax = np.matrix(’100; 100; 100; 100; 100’) p = np.matrix(’3; 2; 7; 6’)
pdisc = np.matrix(’2; 1; 4; 2’) q = np.matrix(’4; 10; 5; 10’) x = cvx.Variable(4)
t1 = cvx.mul_elemwise(p, x)
t2 = cvx.mul_elemwise(p, q) + cvx.mul_elemwise(pdisc, x-q) obj = cvx.Maximize(cvx.sum_entries(cvx.min_elemwise(t1, t2))) cons = [x >= 0, A*x <= cmax]
prob = cvx.Problem(obj, cons) prob.solve()
r = cvx.min_elemwise(t1, t2).value
totr = sum(r)
avgPrice = r / x.value print x.value
print r print totr print avgPrice
using Convex A = [ 1 2 0 1;
0 0 3 1;
0 3 1 1;
2 1 2 5;
1 0 3 2];
cmax = [100;100;100;100;100];
p = [3;2;7;6];
pdisc = [2;1;4;2];
q = [4; 10 ;5; 10];
x = Variable(4);
r = min(p.*x, p.*q + pdisc.*(x - q));
totr = sum(r);
constraints = [x >= 0, A*x <= cmax];
p = maximize(totr, constraints);
solve!(p);
avgPrice = evaluate(r)./evaluate(x);
println(x.value) println(evaluate(r)) println(evaluate(totr)) println(avgPrice)
The result of the code is x =
4.0000 22.5000 31.0000 1.5000
r =
12.0000 32.5000 139.0000 9.0000
totr = 192.5000
avgPrice = 3.0000 1.4444 4.4839 6.0000
We notice that the 3rd activity level is the highest and is also the one with the highest basic price.
Since it also has a high discounted price its activity level is higher than the discount quantity level and it produces the highest contribution to the total revenue. The 4th activity has a discounted price which is substantially lower then the basic price and its activity is therefore lower than the discount quantity level. Moreover it requires the use of a lot of resources and therefore its activity level is low.
3.5 Minimizing the ratio of convex and concave piecewise-linear functions. We consider the problem minimize maxi=1,...,m(aTi x + bi)
mini=1,...,p(cTi x + di) subject to F x g,
with variable x∈ Rn. We assume that cTi x+di> 0 and maxi=1,...,m(aTi x+bi)≥ 0 for all x satisfying F x g, and that the feasible set is nonempty and bounded. This problem is quasiconvex, and can be solved using bisection, with each iteration involving a feasibility LP. Show how the problem can be solved by solving one LP, using a trick similar to one described in§4.3.2.
Solution. We will show that the problem is equivalent to the optimization problem minimize max
i=1,...,m(aTi y + bit) subject to min
i=1,...,p(cTi y + dit)≥ 1 F y gt
t≥ 0
(1)
with variables y, t. This can be further expressed as an LP using the standard tricks, by introducing
an additional variable u:
minimize u
subject to aTi y + bit≤ u, i = 1, . . . , m cTi y + dit≥ 1, i = 1, . . . , p F y gt
t≥ 0.
To show that (1) is equivalent to the problem in the assignment, we first note that t > 0 for all feasible (y, t). Indeed, the first constraint implies that (y, t) 6= 0. We must have t > 0 because otherwise F y 0 and y 6= 0, which means that y defines an unbounded direction in the polyhedron {x | F x g}, contradicting the assumption that this polyhedron is bounded. If t > 0 for all feasible y, t, we can rewrite problem (1) as
minimize t max
i=1,...,m(aTi(y/t) + bi) subject to min
i=1,...,p(cTi(y/t) + di)≥ 1/t F (y/t) g
t≥ 0.
(2)
Next we argue that the first constraint necessarily holds with equality at the optimum, i.e., the optimal solution of (2) is also the solution of
minimize t max
i=1,...,m(aTi(y/t) + bi) subject to min
i=1,...,p(cTi(y/t) + di) = 1/t F (y/t) g
t≥ 0.
(3)
To see this, suppose we fix y/t in (2) and optimize only over t. Since maxi(aTi (y/t) + bi) ≥ 0 if F (y/t) ≤ g, we minimize the cost function by making t as small as possible, i.e., choosing t such that
i=1,...,pmin (cTi (y/t) + di) = 1/t.
The final step is to substitute this expression for the optimal t in the cost function of (3) to get
minimize
i=1,...,mmax (aTi(y/t) + bi)
i=1,...,pmin (cTi(y/t) + di) subject to F (y/t) g
t≥ 0.
This is the problem of the assignment with x = y/t.
3.6 Two problems involving two norms. We consider the problem minimize kAx − bk1
1− kxk∞, (4)
and the very closely related problem
minimize kAx − bk21
1− kxk∞. (5)
In both problems, the variable is x ∈ Rn, and the data are A ∈ Rm×n and b ∈ Rm. Note that the only difference between problem (4) and (5) is the square in the numerator. In both problems, the constraint kxk∞< 1 is implicit. You can assume that b /∈ R(A), in which case the constraint kxk∞< 1 can be replaced withkxk∞≤ 1.
Answer the following two questions, for each of the two problems. (So you will answer four questions all together.)
(a) Is the problem, exactly as stated (and for all problem data), convex? If not, is it quasiconvex?
Justify your answer.
(b) Explain how to solve the problem. Your method can involve an SDP solver, an SOCP solver, an LP solver, or any combination. You can include a one-parameter bisection, if necessary.
(For example, you can solve the problem by bisection on a parameter, where each iteration consists of solving an SOCP feasibility problem.)
Give the best method you can. In judging best, we use the following rules:
• Bisection methods are worse than ‘one-shot’ methods. Any method that solves the problem above by solving one LP, SOCP, or SDP problem is better than any method that uses a one-parameter bisection. In other words, use a bisection method only if you cannot find a ‘one-shot’ method.
• Use the simplest solver needed to solve the problem. We consider an LP solver to be simpler than an SOCP solver, which is considered simpler than an SDP solver. Thus, a method that uses an LP solver is better than a method that uses an SOCP solver, which in turn is better than a method that uses an SDP solver.
Solution. First we discuss the problem
minimize kAx − bk1
1− kxk∞.
(a) This problem is in general not convex. As an example, take n = 1, A =
"
2 0
#
, b =
"
1 0.01
# . The objective is
f (x) = |2x − 1| + 0.01 1− |x| ,
which is not convex (as can be easily seen by plotting the function). However, the problem is quasiconvex: for α≥ 0, the sublevel sets
Sα = {x | kAx − bk1/(1− kxk∞)≤ α}
= {x | kAx − bk1+ αkxk∞≤ α}
are convex. For α < 0 the α-sublevel set is empty, hence convex.
(b) The problem can be solved using a one-parameter bisection and an LP solver. At each iteration we solve the feasibility problem
find x
subject to kAx − bk1/(1− kxk∞)≤ α
(with domain{x | kxk∞< 1}) for fixed α. The assumption b 6∈ R(A) implies that the optimal value is positive, so we only have to consider positive values of α. The feasibility problem can be written in convex form
find x
subject to kAx − bk1+ αkxk∞≤ α.
(Note that if α > 0 the constraint implies the implicit constraintkxk∞< 1.) It can be further rewritten as an LP feasibility problem
find x
subject to −y1 x y1
−z Ax − b z 1Tz + αy≤ α with variables x∈ Rn, y∈ R, z ∈ Rm.
In fact, we can do better and pose the problem as a single LP. We first note that by introducing an auxiliary scalar variable t we can formulate the problem as
minimize kAx − bk1/t subject to t +kxk∞≤ 1
with an implicit constraint t > 0. A change of variables y = x/t, z = 1/t gives a convex problem
minimize kAy − bzk1
subject to 1 +kyk∞≤ z.
(Note that the constraint implies z > 0.) This problem now reduces to an LP minimize 1Tu
subject to −u Ay − bz u
−v1 y v1 1 + v≤ z with variables u∈ Rm, y∈ Rn, z∈ R, v ∈ R.
Next we consider the problem
minimize kAx − bk21
1− kxk∞.
(a) This problem is convex. The objective is the composition of the function Φ(s, t) =
( s2/(1− t) s ≥ 0
with domain R× [0, 1), with the functions g1(x) = kAx − bk1 and g2(x) = kxk∞. Since Φ is convex, and nondecreasing in each argument, and g1 and g2 are convex, the composition Φ(g1(x), g2(x)) is a convex function. (We had to extend the function φ as zero for s < 0 in order to claim that Φ is nondecreasing in each argument.)
As another proof, one can note that the epigraph of the function is a convex set, since kAx − bk21
1− kxk∞ ≤ t, kxk∞< 1 ⇐⇒ kAx − bk21
t +kxk∞≤ 1, t > 0
and the left-hand side of the second inequality is a convex function, jointly in x and t. (The functionkyk21/t is convex because it is the perspective of the convex functionkzk21.)
(b) The problem can be written as an SOCP or SDP. With the assumption that b /∈ R(A), it is equivalent to the problem:
minimize y
subject to s2/(1− t) ≤ y kAx − bk1 ≤ s kxk∞≤ t t≤ 1.
The second and third constraints can be reformulated as linear inequalties. The first constraint is equivalent to the linear matrix inequality
"
1− t s
s y
#
0
and also (using the observation in problem T4.26) to a second-order cone constraint
If we use the SOCP formulation we obtain minimize y
The SDP formulation is
minimize y
There are several variations on this; for example we can just substitute 1Tz for s in the linear matrix inequality.
3.7 The illumination problem. In lecture 1 we encountered the function f (p) = max
i=1,...,n| log aTi p− log Ides| where ai ∈ Rm, and Ides> 0 are given, and p∈ Rm+.
(a) Show that exp f is convex on{p | aTi p > 0, i = 1, . . . , n}.
(b) Show that the constraint ‘no more than half of the total power is in any 10 lamps’ is convex (i.e., the set of vectors p that satisfy the constraint is convex).
(c) Show that the constraint ‘no more than half of the lamps are on’ is (in general) not convex.
Solution. To simplify the notation, we assume that Ides = 1 (if not, we can simply redefine aij
as aij/Ides). We write Ii = aTi p (in the notation of page 1-6 of the lecture notes), so the objective function can be written as
f (p) = max
i | log aTi p|.
The domain of f is
{p | aTi p > 0, i = 1, . . . , n}.
(a) First note that
| log(aTi p)| = max{log aTi p, log(1/aTi p)}
= log max{aTi p, 1/aTi p}, so we can write the objective function as
f (p) = log max
i max{aTi p, 1/aTi p}.
Both aTi p and 1/aTi p are convex on dom f , and therefore maximax{aTip, 1/aTi p} is a convex function. In other words exp f is convex.
(b) This constraint can be expressed as
l
X
i=1
p[i]− 0.5
m
X
i=1
pi ≤ 0
where p[i] is the ith largest component of p (see page 4-11 of the lecture notes). The first term on the left side is the power in the l lamps with the highest power, the second term is one half of the total power. The function on the left hand side is convex, since it is the sum of Pl
i=1p[i], which is convex, and a linear function,
(c) Consider two solutions p1 and p2 that satisfy the constraint. In the first solutions, the first m/2 lamps are on and the rest is off (i.e., the first m/2 components of p1 are positive and the rest is zero); in the second solution the first m/2 lamps are off and the rest is on (i.e., the first m/2 components of p2 are zero, and the rest is positive). The number of nonzero components in a convex combination of p1 and p2 will be m, i.e., the convex combination does not satisfy
3.8 Schur complements and LMI representation. Recognizing Schur complements (see §A5.5) often helps to represent nonlinear convex constraints as linear matrix inequalities (LMIs). Consider the function
f (x) = (Ax + b)T(P0+ x1P1+· · · + xnPn)−1(Ax + b) where A∈ Rm×n, b∈ Rm, and Pi= PiT ∈ Rm×m, with domain
dom f ={x ∈ Rn| P0+ x1P1+· · · + xnPn 0}.
This is the composition of the matrix fractional function and an affine mapping, and so is convex.
Give an LMI representation of epi f . That is, find a symmetric matrix F (x, t), affine in (x, t), for which
x∈ dom f, f(x) ≤ t ⇐⇒ F (x, t) 0.
Remark. LMI representations, such as the one you found in this exercise, can be directly used in software systems such as CVX.
Solution. The epigraph of f is the set of points (x, t) that satisfy P0+ x1P1+· · · + xnPn 0 and (Ax + b)T(P0+ x1P1+· · · + xnPn)−1(Ax + b)≤ t.
Using Schur complements, we can write the second inequality as
"
t (Ax + b)T
(Ax + b) P0+ x1P1+· · · + xnPn
#
0.
This a linear matrix inequality in the variables x, t, i.e., a convex constraint.
3.9 Complex least-norm problem. We consider the complex least `p-norm problem minimize kxkp
subject to Ax = b,
where A ∈ Cm×n, b ∈ Cm, and the variable is x ∈ Cn. Here k · kp denotes the `p-norm on Cn, defined as
kxkp =
n
X
i=1
|xi|p
!1/p
for p≥ 1, and kxk∞= maxi=1,...,n|xi|. We assume A is full rank, and m < n.
(a) Formulate the complex least `2-norm problem as a least `2-norm problem with real problem data and variable. Hint. Use z = (<x, =x) ∈ R2n as the variable.
(b) Formulate the complex least `∞-norm problem as an SOCP.
(c) Solve a random instance of both problems with m = 30 and n = 100. To generate the matrix A, you can use the Matlab command A = randn(m,n) + i*randn(m,n). Similarly, use b = randn(m,1) + i*randn(m,1) to generate the vector b. Use the Matlab command scatter to plot the optimal solutions of the two problems on the complex plane, and comment (briefly) on what you observe. You can solve the problems using the CVX functions norm(x,2) and norm(x,inf), which are overloaded to handle complex arguments. To utilize this feature, you will need to declare variables to be complex in the variable statement. (In particular, you do not have to manually form or solve the SOCP from part (b).)
Solution.
(a) Define z = (<x, =x) ∈ R2n, sokxk22 =kzk22. The complex linear equations Ax = b is the same as<(Ax) = <b, =(Ax) = =b, which in turn can be expressed as the set of linear equations
" Thus, the complex least `2-norm problem can be expressed as
minimize kzk2 (This is readily solved analytically).
(b) Using epigraph formulation, with new variable t, we write the problem as minimize t
This is an SOCP with n second-order cone constraints (in R3).
(c) % complex minimum norm problem
%
randn(’state’,0);
m = 30; n = 100;
% generate matrix A
Are = randn(m,n); Aim = randn(m,n);
bre = randn(m,1); bim = randn(m,1);
A = Are + i*Aim;
b = bre + i*bim;
% 2-norm problem (analytical solution) Atot = [Are -Aim; Aim Are];
btot = [bre; bim];
z_2 = Atot’*inv(Atot*Atot’)*btot;
x_2 = z_2(1:100) + i*z_2(101:200);
% 2-norm problem solution with cvx cvx_begin
variable x(n) complex minimize( norm(x) )
A*x == b;
cvx_end
% inf-norm problem solution with cvx cvx_begin
variable xinf(n) complex minimize( norm(xinf,Inf) ) subject to
A*xinf == b;
cvx_end
% scatter plot figure(1)
scatter(real(x),imag(x)), hold on,
scatter(real(xinf),imag(xinf),[],’filled’), hold off, axis([-0.2 0.2 -0.2 0.2]), axis square,
xlabel(’Re x’); ylabel(’Im x’);
The plot of the components of optimal p = 2 (empty circles) and p =∞ (filled circles) solutions is presented below. The optimal p =∞ solution minimizes the objective maxi=1,...,n|xi| subject to Ax = b, and the scatter plot of xi shows that almost all of them are concentrated around a circle in the complex plane. This should be expected since we are minimizing the maximum magnitude of xi, and thus almost all of xi’s should have about an equal magnitude |xi|.
−0.2 −0.15 −0.1 −0.05 0 0.05 0.1 0.15 0.2
−0.2
−0.15
−0.1
−0.05 0 0.05 0.1 0.15 0.2
Re x
Im x
3.10 Linear programming with random cost vector. We consider the linear program minimize cTx
subject to Ax b.
Here, however, the cost vector c is random, normally distributed with mean E c = c0 and covariance E(c− c0)(c− c0)T = Σ. (A, b, and x are deterministic.) Thus, for a given x∈ Rn, the cost cTx is a (scalar) Gaussian variable.
We can attach several different meanings to the goal ‘minimize cTx’; we explore some of these below.
(a) How would you minimize the expected cost E cTx subject to Ax b?
(b) In general there is a tradeoff between small expected cost and small cost variance. One way
(b) In general there is a tradeoff between small expected cost and small cost variance. One way