Lecture 11: 0-1 Quadratic Program and Lower Bounds

Lecture 11: 0-1 Quadratic Program and

Lower Bounds

(3 units)

Outline

I _{Problem formulations}

I Reformulation: Linearization & continuous relaxation

I Branch & Bound Method framework

I Simple bounds, LP bound and semidefinite relaxation

Problem formulation

I Standard form with 0-1 variables: (0-1QP) min

x∈{0,1}nf(x) = 1 2x

T_Qx+cT_x whereQ is an n ×n symmetric matrix and c ∈Rn_.

I Homogenous form:

(0-1QPh) min x∈{0,1}nx

T_Qx.

I _{Binary variables:}

(BQP) min x∈{−1,1}nx

T_Qx+cT_x. Transformation: xi = 1₂(yi+ 1).

I Homogenous form with binary variables: (BQPh) min

x∈{−1,1}nx T_Qx,

I _{(BQP)⇒ (BQPh) with}

Q :=

µ

0 1 2cT 1

2c Q

¶

, x:= (±1,xT)∈ {−1,1}n+1.

Max-Cut problem

I Consider a graph G = (E,V) with vertex setV ={1, . . . ,n}

and edge set E ={ij |1≤i <j ≤n}. For every edge ij ∈E, there is an associated weight wij.

I Cut: For a given set S ⊆V, a cutδ(S) is the set of all edges with one endpoint inS and the other inV \S, and the weight of cut δ(S) isP_ij∈δ(S)wij.

I Max-Cut: find a cut δ(S) with maximum weight.

I Binary quadratic problem: (Max-Cut) max1

2

X

1≤i<j≤n

wij(1−xixj)

s.t. x ∈ {−1,1}n.

I x ∈ {−1,1}n _{⇔ S ={i ∈V |x}

i = 1} andV \S =

Linearization Method

I _{0-1 Quadratic problem:}

(P) min

x∈{0,1}n Q(x) = n

X

i=1

cixi +

X

1≤i<j≤n

qijxixj.

I For x_i,x_j ∈ {0,1},y_ij =x_ix_j iff

yij = max{xi +xj−1,0}, yij ∈ {0,1}, or

y_ij = min{x_i,x_j}, y_ij ∈ {0,1}.

I (P) is equivalent to the following 0-1 linear integer program: min

x,y n

X

i=1

cixi +

X

(i,j)∈I+

qijyij +

X

(i,j)∈I−

qijyij

s.t. yij ≤xi, yij ≤xj, (i,j)∈I− (qij <0)

yij ≥xi +xj −1, (i,j)∈I+ (qij ≥0),

xi ∈ {0,1}, i = 1, . . . ,n,

y_ij ∈ {0,1}, 1≤i <j ≤n.

I A polynomially solvable case: q_ij ≤0: min

x,y n

X

i=1

cixi +

X

1≤i<j≤n

qijyij

s.t. y_ij ≤x_i, 1≤i <j ≤n

y_ij ≤x_j, 1≤i <j ≤n

xi,xj,yij ∈ {0,1}, 1≤i <j ≤n.

The constraint matrix is totally unimodular!

Continuous relaxation

I Consider the continuous relaxation of (P): ( ¯P) min

x∈[0,1]n Q(x) = n

X

i=1

cixi +

X

1≤i<j≤n

qijxixj. Then, at least one of the optimal solutions of ( ¯P) is located at an extreme point of [0,1]n_{. Thereforev(P) =v( ¯P).}

I Unfortunately, the objective function of ( ¯P) isnonconvex and

nonconcave.

I Define

Q_p(x) =

n

X

i=1

c_ix_i+xTQx−pxTx+peTx.

Qp(x) =Q(x) for x ∈ {0,1}n. For largep,Qp(x) is a

concave function. Thus, (P) is equivalent to the concave minimization problem:

(Pc) min

x∈[0,1]n Qp(x)

Branch and Bound Framework

I _{Computing lower bound;} I Branching on x_i = 0 or x_i = 1;

Basic lower bounding methods

I Simple lower bounds

I Continuous relaxation

I LP relaxation

I Lagrangian relaxation & SDP relaxation

Simple bounds

I Lower bound 1. Let

Q(x) =

n

X

i=1

cixi +1₂xTQx =

n

X

i=1 ˜

cixi +1₂xTQx˜ .

I An obvious lower bound of Q(x) over {0,1}n _is:

LB_s1 = 1

2 n

X

i=1

X

j6=i

min(qij,0) + n

X

i=1

min(ci +1₂qii,0).

I Lower bound 2. An improved simple lower bound is derived

by noting that: sincex ≥0, if ˜Qx ≥a, then 1₂xT_Qx˜ _≥ 1 2aTx.

I _{Let ˜Qi denote theith row of ˜Q. Then}

a_i = min x∈{0,1}n

˜

Q_ix=X

j6=i

I So

min x∈{0,1}n

1 2x

T_{Qx˜ + ˜c}T_x

≥ min

x∈{0,1}n( 1 2a+ ˜c)

T_x

= n

X

i=1

min{c_i +1 2qii+

1 2

X

j6=i

min(q_ij,0),0}

= LB_s2.

It is easy to show thatLB2

s is better thanLBs1, i.e.,

LB_s2 ≥LB_s1.

Continuous relaxation

I Since ˜Q(x) = 1₂xT_{(Q+diag(u))x+ (c−}1

2u)Tx takes the same value on {0,1}n _{asQ(x), it is natural to compute a} lower bound via solving the continuous relaxation:

( ¯P) β(u) = min x∈[0,1]n

1 2x

T_{(Q+diag(u))x+ (c−}1 2u)

T_x.

I Some observations:

I Ifu_i’s are large enough, then ˜Q=Q+diag(u) will be

diagonally dominant (thus positive definite).

I u=λ_mine is an obvious choice to make ˜Q positive semidefinite

(but not necessarily the best one).

I The optimal solution to ( ¯P) tends to (1

2,12, . . . ,12)T asui’s are increased.

I Consider a small example of (P) where

Q =

µ

1 −3

−3 −1

¶ , c =

µ

1 1

¶ .

I For this example, we havex∗ _{= (1,1)}T _{with Q(x}∗_{) =−1.}

The two simple bounds for this problem are: LB_s1=−3 and

LB2

s =−1.

I The eigenvalues of Q is (−2,4).

0 0.2

0.4 0.6

0.8 1

0 0.2 0.4 0.6 0.8 1 −1 −0.5 0 0.5 1 1.5

0 0.2

0.4 0.6

0.8 1

0 0.2 0.4 0.6 0.8 1 −1.5 −1 −0.5 0 0.5 1 1.5

Figure: u=λmine,xu = (0.8604,1)T,β(u) =−1.0406.

0 0.2

0.4 0.6

0.8 1

0 0.2 0.4 0.6 0.8 1 −5 −4 −3 −2 −1 0 1 2

0 0.2

0.4 0.6

0.8 1

0 0.2 0.4 0.6 0.8 1 −25 −20 −15 −10 −5 0 5

Figure: u= 100e,xu= (0.5003,0.5101)T,β(u) =−24.7551.

I Another way of choosing u is to find au∗ _{such that}

β(u∗) = max{β(u)|(Q−diag(u))º0, u ∈Rn}.

I _{The above problem is equivalent to a semidefinite quadratic}

LP relaxation

I The continuous relaxation of the 0-1 linearized problem is a linear program (yij =xixj):

min x,y

X

1≤i<j≤n

qijyij +

n

X

i=1

(ci +1₂qii)xi

s.t. yij ≤xi, yij ≤xj, 1≤i <j ≤n, qij <0,

xi +xj −1≤yij, 1≤i <j ≤n, qij >0,

0≤xi ≤1, i = 1, . . . ,n,

y_ij ≥0, 1≤i <j ≤n.

SDP relaxation

I ConsiderQ(x) = 1

2xTQx+cT.

I The Lagrangian dual is:

v(D)

= max

λ∈Rnd(λ) = max

λ∈Rn_xinf_∈Rn{ 1 2x

T_{[Q+ 2diag(λ)]x+c}T_x−eT_λ}

= max

(λ,τ)∈Rn+1 −τ

s.t. 1

2x

T_{[Q+ 2diag(λ)]x+c}T_x−eT_{λ≥ −τ}

I 1

2xT[Q+ 2diag(λ)]x+cTx−eTλ≥ −τ, x ∈Rn ⇔ (xT,t)

µ

Q+ 2diag(λ) c

cT 2τ −2eTλ

¶ µ

x t

¶ ≥0 (t,x)∈Rn+1,

⇔ _µ

Q+ 2diag(λ) c

cT _2τ−2eT_λ

¶ º0,

Variable fixation

I Let x∗ _{denote the optimal solution of (P). Let}

li =ci +

1 2qii+

X

j6=i

min(0,qij),

u_i =c_i +1

2qii +

X

j6=i

max(0,q_ij).

(i) Ifl_i >0 thenx∗

i = 0;

Fixing variable by outerbox

I Equivalent perturbed problem: (P_µ) min

x∈{0,1}nfµ(x) = 1 2x

T_{(Q+µI)x+ (c−}1 2µe)

T_x,

whereµ≥0,I is then×n identity matrix and

e = (1, . . . ,1)T_.

The shape of contour changes when µchanges. The

properties of (Pµ), e.g., the conditional number, change when µ changes.

−2 −1 0 1 2 3 −4

−3 −2 −1 0 1

x0(µ)

µ=0

µ=1

µ=3 (1,1)

(1,0) (0,1)

(0,0)

µ=5

µ=10 x

2

x

I _{Ellipsoid contour: Let x}0 _{∈ {0,1}}n_.

Eµ={x|fµ(x) =f(x0).

I Let the minimum box that contains ellipsoid Eµbe B = [a,b],

wherea=x0_{−s,b =x}0_{+s. Then}

I Let x∗ _{be an optimal solution to (P). Then}

(i) Ifb_i <1 for some i, then x∗

i = 0;

(ii) Ifai >0 for some i, then xi∗= 1.

−1.5 −1 −0.5 0 0.5 1 1.5 2 −1.5

−1 −0.5 0 0.5 1 1.5

(0,0)

Eµ(˜v)

x0_(µ)

(0,1) (1,1)

Bµ(˜v)