odonnell-slides.pptx

How to refute a random CSP

Sarah R. Allen Ryan O’Donnell David Witmer

Average-case complexity of CSPs.

E.g., random 3SAT.

Random 3SAT

n variables, m = m(n) constraints,

each an OR of 3 uniformly random literals

m

4.267n

satisfiable (w.v.h.p.) unsatisfiable (w.v.h.p.)

Algorithmic task:

find a satisfying assignment

Algorithmic task:

What is this “refutation” task?

1. Algorithm should (w.h.p.) output a proof,

encoded in some formal language like ZFC, that the CSP instance is unsatisfiable.

2. Algorithm should output “unsatisfiable” or “no comment”. It should never be wrong, and it should output “unsatisfiable” w.h.p.

Potential example:

Algorithm solves an LP/SDP relaxation,

Random 3SAT

n variables, m = m(n) constraints,

each an OR of 3 uniformly random literals

m

4.267n

find a sat. assignment find a refutation

3.52n

provably doable

Random 3SAT

n variables, m = m(n) constraints,

each an OR of 3 uniformly random literals

m

4.267n

find a sat. assignment find a refutation

Perhaps when, say, m = n1.4_,

there’s no poly-time refutation algorithm.

Cool! An efficient way to randomly generate simple, hard-to-solve algorithmic tasks!

Potential application 1:

Cryptography. [Gol00, ABW09, Applebaum…]

Potential application 2:

Hardness of learning.

For applications, investigating random CSP(P)

for different predicates P matters.

E.g., for certain kinds of k-ary predicates P,

if refuting random CSP(P) is hard when m = nC_…

App. 1: Gives evidence for PRGs in NC0

with stretch n ↦ nC

App. 2: Shows hardness of PAC-learning concept classes “related to” P,

Refuting random CSPs is

heavily studied for kSAT,

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

New result for kSAT, when k ≥ 5 is odd: we refute provided m ≫ nk/2_;

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Running time is nO(k)_.

The algorithm can be

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Our main theorem:

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

If you take m ≫ nk/2_{, we provide “strong refutation”:}

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

3SAT example: P = OR₃ = 001 010 100 011 101 110 111 Does it support a 2-wise

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

3SAT example: P = OR₃ =

001 010 100 011 101 110 111

Does it support a 2-wise

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

3SAT example: P = OR₃ = 001 010 100 011 101 110 111 Does it support a 2-wise

uniform distribution? Yes.

Does it support a 3-wise uniform distribution? No.

∴ cmplx(OR₃) = 3.

We can refute once

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

2-out-of-4-SAT: P =

0011 0101 0110 1001 1010 1100

Does it support a 2-wise uniform distribution? No.

(If it did, by symmetrization that distribution could be the uniform distribution over weight-2, length-4 strings,

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

2-out-of-4-SAT: P =

0011 0101 0110 1001 1010 1100

Does it support a 2-wise uniform distribution? No.

∴ cmplx(P) = 2.

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

Does it support a 2-wise uniform distribution? No.

∴ cmplx(P) = 2.

We can refute random instances once m ≫ n. The exact same thing holds for any

j-out-of-k-SAT.

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

[DLS14]: A very strong conjecture implying that for many k-ary predicates P, random

CSP(P) instances with nω_k→∞(1) _constraints

are not refutable in polynomial time.

Instantiated with 3 families (P_k),

derived 3 hardness-of-learning results.

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution.

[DLS14]: A very strong conjecture implying that for many k-ary predicates P, random

CSP(P) instances with nω_k→∞(1) _constraints

are not refutable in polynomial time.

Instantiated with 3 families (P_k),

derived 3 hardness-of-learning results.

A condition similar to m ≫ n cmplx(P) / 2 also arises

in work on finding satisfying assignments

in random CSPs with a planted solution [FPV14].

Warning / Remark

Nevertheless, I assure you:

solving planted CSPs & refuting random CSPs

A poly-time algorithm that, for any k-ary predicate P, refutes (whp) uniformly random

CSP(P) instances, provided m ≫ n cmplx(P) / 2_.

Plan for the rest of the talk

Part 1: Strong refutation for kXOR Part 2: Strong refutation for any

k-CSP, provided m ≫ nk/2

Given a random kXOR CSP with m ≫ nk/2_,

want to certify “OPT ≤ ½·m +o(m)” (whp).

• _{[CCF10] strongly refute any CSP with m ≫ n}⌈k/2⌉.

• _{For kXOR specifically, m ≫ n}⌈k/2⌉ follows pretty

easily from approximation algs literature.

• _{For 3SAT, strong refutation provided m ≫ n}1.5

achieved by [CGL07]. Messy, but if you stare at

it long enough, it’s clear it generalizes to 3XOR. • _{This, as well as the generalization to kXOR,}

Given a random kXOR CSP with m ≫ nk/2_,

want to certify “OPT ≤ ½·m +o(m)” (whp).

• _{The weaker bound m ≫ n}⌈k/2⌉ follows from the

k = 2 case, and a very trivial trick.

• _{The m ≫ n}k/2 _{bound requires adding in a very}

clever trick (from [FGK01,CGL07]).

Given a random 2XOR CSP with m ≫ n, want to certify “OPT ≤ ½·m +o(m)” (whp).

n variables, x₁, …, x_n ∈ {0,1}

m constraints, each like

Given a random 2XOR CSP with m ≫ n, want to certify “OPT ≤ ½·m +o(m)” (whp).

n variables, x₁, …, x_n ∈ {−1,+1}

m constraints, each like

Given a random 2XOR CSP with m ≫ n, want to certify “OPT ≤ o(m)” (whp).

n variables, x₁, …, x_n ∈ {−1,+1}

m constraints, each like

Given a random 2XOR CSP with m ≫ n, want to certify “|OPT| ≤ o(m)” (whp).

n variables, x₁, …, x_n ∈ {−1,+1}

m constraints, each like

Given a random 2XOR CSP with m ≫ n, want to certify “|OPT| ≤ o(m)” (whp).

n variables, x₁, …, x_n ∈ {−1,+1}

m constraints, each like

Let and instead include each

Given a random 2XOR CSP with p ≫ 1/n, want to certify “|OPT| ≤ o(pn2_{)” (whp).}

n variables, x₁, …, x_n ∈ {−1,+1}

m constraints, each like

~pn2

Let A be the random n×n symmetric matrix with entries {−1,0,+1} depending on the x_i,x_j constr.

Given assignment x ∈ {−1,+1}n_{, it has}

Given a random 2XOR CSP with p ≫ 1/n, want to certify “|OPT| ≤ o(pn2_{)” (whp).}

Let A be the random n×n symmetric matrix with entries {−1,0,+1} depending on the x_i,x_j constr.

Given assignment x ∈ {−1,+1}n_{, it has}

|# sat − # unsat| = |xTAx| ≤ n||A||

Done if ||A|| ≤ o(pn) whp. (Can efficiently certify.)

A good old-fashioned random matrix fact.

Part 1: Strong refutation for kXOR

Part 2: Strong refutation for any

k-CSP, provided m ≫ nk/2

m k

m k

Def: Given an assignment x∈{0,1}n_,

the table distribution D_x is the probability distribution on {0,1}k _{given by choosing a}

m k

Def: Given an assignment x∈{0,1}n_,

the table distribution D_x is the probability distribution on {0,1}k _{given by choosing a}

uniformly random row from the above table.

Def: Given an assignment x∈{0,1}n_,

the table distribution D_x is the probability distribution on {0,1}k _{given by choosing a}

uniformly random row from the above table.

Def: An instance I is quasirandom if for all assignments x∈{0,1}n_,

the table distribution D_x is o(1)-close

to the uniform distribution on {0,1}k_.

Fact: If a CSP(P) instance I is quasirandom, then

Given a random k-ary CSP with m ≫ nk/2_,

want to certify it’s quasirandom (whp).

By “Vazirani XOR Lemma”, a table distribution

D_x is o(1)-close to uniform if and only if

D_x has o(1)-correlation with XOR of S coordinates for all ∅ ≠ S ⊆ [k].

Part 1: Strong refutation for kXOR

Part 2: Strong refutation for any

k-CSP, provided m ≫ nk/2

✔

✔

cmplx(P): least t ≥ 2 such that P does not

support a t-wise uniform distribution. Suppose P does not support

Suppose P does not support a t-wise uniform distribution.

⇒ every t-wise uniform distribution is

δ-far from being supported on P

(for some constant δ = δ(P) > 0, by basic LP theory).

Given CSP(P) instance,

instead of certifying quasirandomness

Suppose P does not support a t-wise uniform distribution.

⇒ every t-wise uniform distribution is

δ-far from being supported on P

(for some constant δ = δ(P) > 0, by basic LP theory).

Given CSP(P) instance,

strongly refute S-XOR for all |S| ≤ t. This is doable (whp) provided m ≫ nt/2.

This certifies that all table distributions D_x

Suppose P does not support a t-wise uniform distribution.

⇒ every t-wise uniform distribution is

δ-far from being supported on P

(for some constant δ = δ(P) > 0, by basic LP theory).

Given CSP(P) instance,

strongly refute S-XOR for all |S| ≤ t. This is doable (whp) provided m ≫ nt/2.

This certifies that all table distributions D_x

are o(1)-close to uniform, and hence

(δ−o(1))-far from being supported on P.

Part 1: Strong refutation for kXOR

Part 2: Strong refutation for any

k-CSP, provided m ≫ nk/2

✔

✔

Part 3: Main theorem, refuting random

Open directions

Give evidence that ncmplx(P) /2 is optimal.

I.e., if P supports a (t−1)-wise uniform distribution, try to show various efficient refutation systems fail on random instances with m ≪ nt/2 _constraints.

[BW99]: Resolution fails on kXOR and kSAT.

[Sch08]: SOS fails on kXOR and kSAT.

[BGMT12,TW13,OW14,MWW15]: Sherali–Adams+ fails.

[FPV15]: Certain “statistical algorithms” fail.

[BCK15]: For t=3 (pairwise unif.), “pruned” random