potechin.pptx

(1)

Sum of Squares Lower

Bounds for the Planted

Clique Problem

Aaron Potechin MIT

In collaboration with Boaz Barak, Siu On Chan,

Jonathan Kelner, Raghu Meka, David Steurer, and Avi Wigderson

(2)

Talk Outline

• _{Part I: Planted Clique}

• _{Part II: The Sum of Squares/Lasserre Hierarchy}

• _{Part III: Analyzing the Meka-Wigderson Moment}

(3)

Part I:

(4)

Max-Clique

Largest subset of vertices with all edges present

Classical optimization problem

(5)

Worst Case Complexity of

Max-Clique

• _{NP-hard (made Karp’s list)} • _{NP-hard to approximate by}

-Hastad 99, Zuckerman 06

• _{Unconditional hardness in various models}

-Razborov 85 -…

-Tulsiani 09

(6)

Cliques in G(n,1/2)

• _{Largest clique has size roughly} • _{Easy to find cliques of size .}

(7)

The Planted Clique

Problem

• _{G(n,1/2) + clique(k)}

• _{Jerrum 92, Kucera 95:}

For which k can we find the planted clique?

• _{Best- Alon et al. 98:}

(8)

The Planted Clique

Problem

• _{G(n,1/2) + clique(k)}

• _{Jerrum 92, Kucera 95:}

For which k can we find the planted clique?

• _{Best- Alon et al. 98:}

This 5-clique was planted by adding the red edge.

(9)

Square-root Barrier

• _{Jerrum 92: Can’t do using MCMC}

(Monte-Carlo Markov Chains)

• _{Feige-Krauthgamer 00,03: Using}

(the rth level of the Lovasz-Schrijver hierarchy) -Can if k is

-Can’t if k is

• _{Feldman et al. 12: Can’t do using statistical}

(10)

Why Planted Clique?

• _{Natural algorithmic problem and test bed}

-Spectral algorithms, clustering

• _{Natural average case hardness candidate}

Cryptosystems - [Juels, Peinado 00]

Nash equilibria - [Hazan, Krauthgamer 11]

(11)

• _{Variable for each vertex i in G.} • _{Want if i is not in the clique}

• _{Want if i is in the clique.} • _Equations:

= for all i. = 0 if

= k

• _{These equations are feasible precisely when}

G contains a k-clique.

(12)

Part II:

(13)

The Sum of

Squares/Lasserre Hierarchy

• _{Developed independently by Shor, Nesterov,}

Parrillo, and Lasserre.

• _{Generalization of linear and semidefinite}

programming

• _{Each level gives a more powerful feasibility}

test than the last, rth level takes time.

• _{These tests can often be translated into}

approximation algorithms.

• _{Performance of these feasibility}

(14)

The Power of the Sum of

Squares Hierarchy

• _{Strictly stronger than the Lovasz-Schrijver}

hierarchy and the Sherali-Adams hierarchy

• _{Leading candidate for refuting Khot’s}

Unique Games Conjecture.

• _{Captures the known subexponential time}

algorithm for Unique Games and can solve many proposed gap instances for other

(15)

A Game for the Sum of

Squares Hierarchy

• _{Setup: Base problem is to determine the}

feasibility of a system of polynomial equations over the reals, e.g.

• _{= for all v.} • _{= k}

(16)

A Game for the Sum of

Squares Hierarchy

• _{Two players, Optimist and Pessimist.}

• _{Optimist must claim that the answer is yes}

and give some evidence

• _{Pessimist must try to disprove Optimist’s}

evidence.

• _{Pessimist wins if he/she is able to refute}

(17)

• _{What evidence should be required of Optimist?} • _{Choice 1: Optimist must give the value of all}

variables.

-To win, Optimist must fully solve the problem.

• _{Choice 2: No evidence.}

-To win, Pessimist must prove infeasibility.

• _{We want something in the middle.}

(18)

• _{For the rth level of the SOS hierarchy,}

Optimist must give the expectation values of all monomials up to degree 2r for some

distribution of solutions.

A Game for the Sum of

Squares Hierarchy

Equations for whether G has a triangle:

= for all vertices i of G. = k = 3

= 0 whenever i and j are not adjacent in G.

1

4 3

2

(19)

A Game for the Sum of

Squares Hierarchy

Optimist can give the following expectation values (when r = 1):

E[] = E[] = E[] = E[] =

E[] = E[] = E[] = E[] = 3/4 E[] = E[] = E[] =

E[] = E[] = E[] = 1/2.

This corresponds to taking each of the 4 triangles in G with probability 1/4.

1

4 3

2

(20)

A Game for the Sum of

Squares Hierarchy

Of course, Optimist could try to lie…

For example, Optimist could give the following pseudo-expectation values:

Ẽ[] = Ẽ[] = Ẽ[] = Ẽ[] =

Ẽ[] = Ẽ[] = Ẽ[] = Ẽ[] = 3/4 Ẽ[] = Ẽ[] = Ẽ[] = Ẽ[] = 3/4

Ẽ[] = Ẽ[] = 0. 1

4 3

2

(21)

Detecting Lies

1

4 3

2

G

How can Pessimist detect lies systematically?

1. If the pseudo-expectation values don’t obey the expected equations, it’s bogus!

Let’s check some: (all vertices and edges have pseudo-expectation value 3/4)

3/4 3/4 0 3/4 9/4 3]

(22)

Detecting Lies

1

4 3

2

G

How else can Pessimist detect lies?

2. If some square has negative pseudo-expectation value, it’s bogus!

Ẽ[]

Ẽ[] Ẽ[] Ẽ[] Ẽ[] 2Ẽ[] 2Ẽ[] 2Ẽ[] 2Ẽ[] 2Ẽ[] 2Ẽ[]

3/4 3/4 3/4 3/4 0

(23)

• _{We restrict Pessimist to these two methods.}

• _{Optimist wins if he can come up with a pseudo}

expectation Ẽ which obeys all of the required equations and has nonnegative expectation on all squares

• _{All constraints on Ẽ are convex, so we can find}

Ẽ (if it exists) with semidefinite programming.

(24)

The Moment Matrix

• _{Each f of degree corresponds to a vector} • _Ẽ[f2_{] = f}T_Mf

• _{Constraint that Ẽ is nonnegative on squares is}

satisfied if M is PSD (positive semi-definite)

𝑞

𝑝 _Ẽ[p

𝑀

(25)

• _{Does Pessimist have a general strategy too?}

• _{Yes, a Positivstellensatz (sum of squares) proof of}

infeasibility.

• _{Pessimist must find polynomials f and g of degree}

at most 2r such that:

1. f = 0 by the problem equations 2. g is a sum of squares

3. -1 = f + g

• _{This proves the equations are infeasible}

• _{All constraints on f,g are convex, so we can find f,g}

(if they exist) with semidefinite programming.

(26)

Duality Continued

• _{Elementary fact: Optimist and Pessimist cannot}

both have a winning strategy.

• _{Reason: apply Ẽ to the equation -1 = f + g}

• _{By convex duality, in virtually all cases, either}

(27)

Using the Sum of Squares

Hierarchy

Pessimist Wins

Optimist Wins Problem is Feasible

• _{The SOS hierarchy tells us approximately when}

our equations are feasible.

• _{If we increase r, it becomes harder for Optimist to}

(28)

Obtaining Approximation

Algorithms

• _{Let’s say we are trying to optimize some}

parameter k. If we have a rounding algorithm which turns a pseudo-expectation Ẽ into an

actual solution (at some cost to k), this gives an approximation algorithm.

Pessimist Wins

Optimist Wins Problem is Feasible

Ẽ A Solution

(29)

Two views of the Sum of

Squares Hierarchy

• _{Relaxation view: Solving our equations exactly}

is hard, so we relax this to finding a suitable Ẽ.

• _{Key questions: When will this relaxation be}

feasible? Can we round from Ẽ to a nearly optimal solution?

• _{Proof complexity view: The sum of squares}

hierarchy gives a proof system

(Postivistellensatz proofs) to show infeasibility.

• _{Key questions: How high does r have to be to}

(30)

Sum of Squares and

Planted Clique

• _{Essential question: On a}_random_{graph, for}

which k does Optimist win the SOS(r) game?

• _{If Optimist wins, SOS(r) cannot determine}

whether or not a clique of size k was planted.

• _{If Pessimist wins, SOS(r) can determine}

whether or not a clique of size k was planted.

• _{Lower bound strategy: Find a}

(31)

Previous Work

• _{Previously, no non-trivial lower bounds were}

known for levels.

• _{In 2013, Raghu Meka and Avi Wigderson}

(32)

Our Results

• _{Using Meka and Wigderson’s moment matrix}

M, we recover a weaker version of the claimed result.

• _{In particular, the rth level of the Lasserre/SOS}

hierarchy cannot solve the planted clique problem unless

• _Note:_{Meka and Wigderson’s moment matrix}

(33)

Part III:

(34)

The MW Moments

• _{Idea: Let d = 2r and give each d-clique the}

same weight.

Definition: Define to be the number of d-cliques containing V.

Definition: Define and set whenever .

(35)

Our goal

• _{We must show that that the following}

moment matrix M is PSD:

𝑥_𝑉

𝑥_𝑊

[] _|𝑉_|₌_|𝑊_|₌𝑟

(36)

Estimating the entries of M

• _{Think of d as a constant and n >> k >> 1.} • _{If V is a clique,}

(37)

Picture of M for r = 1 (d = 2)

1 2 3 4 5 6 1

2 3 4 5 6

0 or

Row i, Column i

(38)

Picture of M for r = 2 (d = 4)

0 or

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 0 or 0 or

Row ij, Column ij Nonzero if

Row ij, Column ik Nonzero if

i j

i j k

Row ij, Column kl Nonzero if i

j

k

(39)

Difficulties in Analyzing M

• _{Difficulty #1: Only know nonzero entries}

approximately.

• _{Fix: Separate out the discrepancy into a}

(40)

Picture of M - Δ

0 or

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 0 or 0 or

Row ij, Column ij Nonzero if

Row ij, Column ik Nonzero if

i j

i j k

j

k

(41)

Difficulties in Analyzing M - Δ

• _{Difficulty #2: M - Δ has many zero rows and}

columns

• _{Fix: Fill in the zero rows/columns of M – Δ} • _{Calling the resulting matrix M’, the smallest}

(42)

Filling in the zero rows/columns

• _{How should we fill in the zero rows/columns}

of M – Δ?

• _{Idea: Only look at edges between row index}

(43)

Picture of M’

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 0 or 0 or

Row ij, Column ik Nonzero if j k

j

k

(44)

Decomposition of M’

• _{We decompose M’ as M’ = E + R} • _{E is the expected value of M’}

• _{R is the random part of M’}

• _{We will show that E is strongly PSD i.e. the}

smallest eigenvalue of E

(45)

Picture of E

(46)

Picture and Analysis of E for r = 1

1 2 3 4 5 6 1

2 3 4 5 6

Row i, Column i

Row i, Column j

(47)

Analysis of E for r=2

• _{We can decompose E using the Johnson}

scheme

• _{Idea: E is sum of PSD matrices, including , so}

(48)

Picture of R

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 or or 15

Row ij, Column ik Positive if j k

Row ij, Column kl Positive if i

j

k

(49)

Decomposition of R

• _{We write +}

(50)

Picture of

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56

±

1

(51)

Decomposition of

• _{Difficulty: Structure of is complicated.}

• _{Fix: Partition the entries of based on which}

(52)

Picture of a piece of

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56

±

1

Row ij, Column ik i < j and i < k

(53)

Analysis of

• _{Idea: can be decomposed as a sum of a}

constant number of pieces like the one shown.

(54)

Picture of

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 or

Row ij, Column kl Positive if i

j

k

(55)

Analysis of

(56)

Putting everything together

• _{w.h.p and}

• _{Recall that +}

• _{M’ is strongly PSD so long as} • _{This happens as long as}

• _{The nonzero part of M – Δ is a submatrix of}

M’, so it is strongly PSD.

• _{is small, so the nonzero part of M is strongly}

(57)

Takeaways

• _{The sum of squares hierarchy gives a series of} feasibility tests, each more powerful than the last, whose performance is not well understood. • _{To show that SOS(r) does not certify infeasibility}

of a problem, we must:

• _{1. Give a candidate pseudo-expectation which} obeys all of the equations for the problem.

• _{2. Show that the resulting moment matrix is PSD.} • _{For planted clique, analyzing the MW moment}

(58)

Further Research and Future Work

• _{Can we improve this analysis to prove better}

lower bounds using the MW moments?

(Spoiler: We think the correct bound is around )

• _{Are there pseudo-expectations that give}

better lower bounds than the MW moment? (Spoiler: We think the MW moments can be improved, at least for r = 2 rounds)

• _{Can we get a lower bound of for r rounds}

(59)

Acknowledgements

• _{Thanks to Raghu for letting me use some of his}

(60)

(61)

Appendix:

(62)

Correctness of the MW moments

• _{Recall the equations for the k-clique problem:}

1. = for all vertices i of G. 2. = 0 if

3. = k

• _{We need to check that Ẽ obeys these}

(63)

Correctness of the MW moments

• _{Check #1: .}

• _{Check #2: If g and g has degree at most d}

then .

• _Recall:_{for all whenever f has degree at most}

(64)

Correctness of the MW moments

then .

• _Note:_{If V is not a clique then so .}

(65)