PCP and Inapproximability

PCP and Inapproximability

Irit Dinur

NEC

• Example: the Minimum Vertex Cover function Example: the Minimum Vertex Cover function

• Facts: 1. Best algorithm runs in time (1.21) Facts: 1. Best algorithm runs in time (1.21) ^{n n} [Robson ‘86] [Robson ‘86]

• 2. VC is NP-hard. [Karp ’72] 2. VC is NP-hard. [Karp ’72]

• What about approximation.. Output a vertex cover that’s What about approximation.. Output a vertex cover that’s

“nearly” minimal!

“nearly” minimal!

Minimum Minimum

Vertex Vertex Cover Cover

Given any function..

Given any function..

How complicated is it to compute it?

How complicated is it to compute it?

4

Vertex-Cover:

Vertex-Cover: Given a network of roads. Each road requires a toll Given a network of roads. Each road requires a toll payment.

payment.

Goal: Put up the smallest number of toll booths

Goal: Put up the smallest number of toll booths

7

What do we mean by approximation?

What do we mean by approximation?

Each instance has many solutions, each has a value.

Each instance has many solutions, each has a value.

In optimization, we are seeking the minimal.

In optimization, we are seeking the minimal.

4 5

MIN MIN

An approximation algorithm: finds a solution within a certain neighborhood of MIN

Approx Approx

Example: An algorithm for

Example: An algorithm for Approximating Vertex Cover Approximating Vertex Cover

1. 1. Given G, find a maximal set of edges that do not touch each Given G, find a maximal set of edges that do not touch each other.

other.

2. 2. Add both vertices of each edge to the vertex cover. Add both vertices of each edge to the vertex cover.

7

4 5

An approximation algorithm: finds a solution within a certain neighborhood of MIN

Approximation Approximation

How big is it?

How big is it?

No more than twice No more than twice

the minimum!

the minimum!

This is

This is a a solution: all solution: all edges are covered

edges are covered

Example: An algorithm for

Example: An algorithm for Approximating Vertex Cover Approximating Vertex Cover

1. 1. Given G, find a maximal set of edges that do not touch each Given G, find a maximal set of edges that do not touch each other.

other.

2. 2. Add both vertices of each edge to the vertex cover. Add both vertices of each edge to the vertex cover.

An approximation algorithm: finds a solution within a certain neighborhood of MIN

We’ve seen an approximation algorithm for Vertex- We’ve seen an approximation algorithm for Vertex-

Cover, with approximation factor 2.

Cover, with approximation factor 2.

No more than twice No more than twice

the minimum!

the minimum!

Example: An algorithm for

Example: An algorithm for Approximating Vertex Cover Approximating Vertex Cover

1. 1. Given G, find a maximal set of edges that do not touch each Given G, find a maximal set of edges that do not touch each other.

other.

2. 2. Add both vertices of each edge to the vertex cover. Add both vertices of each edge to the vertex cover.

MIN MIN

x 2

Approx Approx

We’ve seen a factor 2 algorithm.

We’ve seen a factor 2 algorithm.

Q: Is there a factor 1.99 algorithm?

Q: Is there a factor 1.99 algorithm?

3/2 ? 3/2 ? 4/3 ? 4/3 ?

x 3/2

x 4/3 x 1.99

Not unless P=NP,

[AS,ALMSS,BGS,Raz,Hastad,DS]

We don’t know

Approximation

Approximation

Finding the Approximation Threshold Finding the Approximation Threshold

• Rich variety of approximation algorithms. Rich variety of approximation algorithms.

• Various problems with vastly differing approximation Various problems with vastly differing approximation ratios.

ratios.

• PCP theorem: breakthrough, we can prove hardness of PCP theorem: breakthrough, we can prove hardness of approximation.

approximation.

• Moreover, it’s within reach to find the exact Moreover, it’s within reach to find the exact threshold threshold where a problem changes from being easy to hard.

where a problem changes from being easy to hard.

(theoretic and practical importance) (theoretic and practical importance)

• Vertex Cover: Vertex Cover:

• Upper bound: 2-o(1) Upper bound: 2-o(1) [BYE, MS, Hal] [BYE, MS, Hal]

• Best hardness result: Best hardness result:

Thm Thm [DS`02] [DS`02] : NP-hard to approximate to within 1.36. : NP-hard to approximate to within 1.36.

Conjecture

Conjecture : NP-hard to within 2- : NP-hard to within 2-     >0 >0

How to show hardness?

How to show hardness?

Take any NP-hard problem, say SAT, and “reduce”

Take any NP-hard problem, say SAT, and “reduce”

Translating a formula

Translating a formula   into a graph G, s.t. into a graph G, s.t.

VC(G) > k VC(G) > k

  is SAT is SAT VC(G)=k VC(G)=k

  is unSAT is unSAT

The gap-version of the problem The gap-version of the problem

MIN = k

MIN = k Approx < Approx < (1+c) (1+c)   k k

Given a

Given a 1+c 1+c approximation algorithm, approximation algorithm, If MIN = k then Approx < (1+c)

If MIN = k then Approx < (1+c)   k. k .

A 1+c A 1+c approximation algorithm could decide approximation algorithm could decide whether the minimum is

whether the minimum is

• below k below k or or

• above (1+c) above (1+c) k  k

In other words, it would solve the

In other words, it would solve the gap problem gap problem : :

Given a graph G, decide if the minimum VC is

Given a graph G, decide if the minimum VC is

How to show hardness of How to show hardness of

approximation?

approximation?

Take any NP-hard problem, say SAT, and “reduce”

Take any NP-hard problem, say SAT, and “reduce”

Translating a formula

Translating a formula  into a graph G, s.t.  into a graph G, s.t.

Decades, complexity of approximations remains mysterious Decades, complexity of approximations remains mysterious Rich variety of algorithms, hardly any lower-bounds

Rich variety of algorithms, hardly any lower-bounds Why? Why?

VC(G) > k VC(G) > k

  is SAT is SAT VC(G)=k VC(G)=k

  is unSAT is unSAT

VC(G) > (1+c)k

VC(G) > (1+c)k

  is SAT is SAT VC(G)=k VC(G)=k

(x  (x 0 1 0 0 1 0 _{1 1} ,x ,x _{2 2} ,…,x ,…,x _{n n} ) )

Simple: Every SAT assignment translates to a size-k VC for G.

VC(G) > k VC(G) > k

  is SAT is SAT VC(G)=k VC(G)=k

  is unSAT is unSAT

VC(G) > (1+c)k VC(G) > (1+c)k

(x  (x 0 1 0 0 1 0 _{1 1} ,x ,x _{2 2} ,…,x ,…,x _{n n} ) )

Simple: Every SAT assignment translates to a size-k VC for G.

And: Every size-k VC translates to a SAT assignment.

But, far from easy:

Given a VC of size < (1+c)k, how to decode it into a SAT assignment?

“ “ Combinatorial Decoding” Combinatorial Decoding”

P P robabilistically robabilistically C C heckable heckable P P roofs roofs

PCP – a referee’s dream PCP – a referee’s dream

• Background context – Interactive Proofs Background context – Interactive Proofs

• Say you get a paper for refereeing. Say you get a paper for refereeing.

• Select a few random lemmas and verify only them. Select a few random lemmas and verify only them.

• Will this work? – in general, of course not! Will this work? – in general, of course not!

• PCP Theorem PCP Theorem : Every proof can be compiled into : Every proof can be compiled into

“PCP-language” so that by reading only a few bits of

“PCP-language” so that by reading only a few bits of the new PCP proof, correct verification can be

the new PCP proof, correct verification can be achieved with high probability.

achieved with high probability.

• Why is this amazing? Why is this amazing?

• One tiny bug or hole or gap can cause much agony… One tiny bug or hole or gap can cause much agony…

• In a proof usually the correctness of each step depends on In a proof usually the correctness of each step depends on much of what happened before it…

much of what happened before it…

• It seems that if you only read a few bits, you can easily be It seems that if you only read a few bits, you can easily be cheated!

cheated!

x ₁ x ₂ x ₃ x ₄

x ₅ x ₆ x ₇ x ₈

x _n-3 x _n-2 x _n-1 x _n x ₉

x ₁₀ x ₁₁ x ₁₂

x ₁₃ x ₁₄ x ₁₅ x ₁₆

… …

• An NP statement can be written as a 3SAT formula An NP statement can be written as a 3SAT formula

• A proof for its satisfiability is an assignment of 0/1: A proof for its satisfiability is an assignment of 0/1:

• We can verify it clause by clause. We can verify it clause by clause.

1 0 1 0

0 0 1 0

1 0 0 1

1 1 0 0

1 0 1 1

( ( x x 1 ₁ v v x x 3 ₃ v v x x 12 ₁₂ ) ) ( ( x x 1 ₁ v v x x 2 ₂ v v x x 8 ₈ ) ) ( ( x x 7 ₇ v v x x n _n v v x x 10 ₁₀ )… )… ( ( x x 1 ₁ v v x x 15 ₁₅ v v x x 14 ₁₄ ) )

More Concretely

More Concretely

x ₁ x ₂ x ₃ x ₄

x ₅ x ₆ x ₇ x ₈

x _n-3 x _n-2 x _n-1 x _n x ₉

x ₁₀ x ₁₁ x ₁₂

x ₁₃ x ₁₄ x ₁₅ x ₁₆

… …

x ₁

x ₃

x ₁₂ x ₁

x ₂

x ₈ x ₇

x _n x ₁₀

x ₁

x ₁₄ x ₁₅

1 0 1 0

0 0 1 0

1 0 0 1

1 1 0 0

1 0 1 1

( ( x x 1 ₁ v v x x 3 ₃ v v x x 12 ₁₂ ) ) ( ( x x 1 ₁ v v x x 2 ₂ v v x x 8 ₈ ) ) ( ( x x 7 ₇ v v x x n _n v v x x 10 ₁₀ )… )… ( ( x x 1 ₁ v v x x 15 ₁₅ v v x x 14 ₁₄ ) )

• An NP statement can be written as a 3SAT formula An NP statement can be written as a 3SAT formula

• A proof for its satisfiability is an assignment of 0/1: A proof for its satisfiability is an assignment of 0/1:

• We can verify it clause by clause. We can verify it clause by clause.

• Murphy’s law! we detect an “error” only on the last clause Murphy’s law! we detect an “error” only on the last clause

More Concretely

More Concretely

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

=  = ( ( x x

x x

x x

)…( )… ( x x

x x

x x

) )

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

More Concretely More Concretely

 The PCP theorem, gives a “compiler” translating The PCP theorem, gives a “compiler” translating   into into   s.t.

s.t.   admits super-efficient verification. admits super-efficient verification.

 If If   is satisfiable, then so is is satisfiable, then so is  

 If If  is  is not not satisfiable, then every assignment satisfies at satisfiable, then every assignment satisfies at most 99% of most 99% of   ’s clauses. ’s clauses.

Y

Y

Y

=  = (y (y

v v y y

v v y y

)(y )(y

v v y y

v v y y

)(y )( y

v v y y

v v y y

)…(y )…( y

v v y y

v v y y

) )

  is unSAT is unSAT   is <99% SAT is <99% SAT

  is SAT is SAT   is SAT is SAT

1 0 0 1

0 0 1 1

0 1 0 1

0 0 0 0

1 1 1 1

1 0 1 1

0 0 0 1

1

1

0

0

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

=  = ( ( x x

x x

x x

)…( )… ( x x

x x

x x

) )

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

y

More Concretely More Concretely

 Conclusion [AS,ALMSS] “gap-SAT is NP-hard” Conclusion [AS,ALMSS] “gap-SAT is NP-hard”

Given a 3-SAT formula Given a 3-SAT formula   it is NP-hard to decide whether: it is NP-hard to decide whether:

  is satisfiable is satisfiable

 Every assignment satisfies at most 99% of Every assignment satisfies at most 99% of ’s clauses.  ’s clauses.

=  = (y (y

v v y y

v v y y

)(y )(y

v v y y

v v y y

)(y )( y

v v y y

v v y y

)…(y )…( y

v v y y

v v y y

) )

  is unSAT is unSAT   is <99% SAT is <99% SAT

  is SAT is SAT   is SAT is SAT

This is an Inapproximability Result!

This is an Inapproximability Result!

• Hardness for gap-3SAT, shows that it is NP-hard to approximate the Hardness for gap-3SAT, shows that it is NP-hard to approximate the following optimization problem:

following optimization problem:

Max-3-SAT

Max-3-SAT : Given a 3SAT formula, : Given a 3SAT formula, =  = ( ( y y

v v y y

v v y y

)… )… ( ( y y

v v y y

v v y y

), ),

find the maximal fraction of satisfiable clauses.

find the maximal fraction of satisfiable clauses.

• A random assignment easily satisfies 7/8 A random assignment easily satisfies 7/8

• Håstad proved there’s no better algorithm Håstad proved there’s no better algorithm

  is unSAT is unSAT   is <99% SAT is <99% SAT

  is SAT is SAT   is SAT is SAT

Some Brief History Some Brief History

• Context: Interactive proofs Context: Interactive proofs

• Connection to Approximation [FGLSS `91] Connection to Approximation [FGLSS `91]

• PCP Theorem: [AS, ALMSS `92] PCP Theorem: [AS, ALMSS `92]

• Immediately for many problems, e.g. vertex Immediately for many problems, e.g. vertex cover, max-cut, metric-TSP, max-3SAT,

cover, max-cut, metric-TSP, max-3SAT, bounded-degree-clique, … [PY `91]

bounded-degree-clique, … [PY `91]

• Boom of hardness of approximation results Boom of hardness of approximation results

• Emphasis on Emphasis on

• Better PCP parameters Better PCP parameters

• Tight Tight inapproximability results inapproximability results

Tighter Results Tighter Results

• […,BGLR,FK,BS]: […,BGLR,FK,BS]: Better reductions with explicit Better reductions with explicit constants

constants

• [ BGS [ BGS ^{‘95 ‘95} ]: ]: Introduced the Long-Code Introduced the Long-Code . . e.g. for VertexCover:

e.g. for VertexCover: 1.068 1.068 , for Max-CUT: , for Max-CUT: 1.014 1.014

• [Håstad [Håstad 96-97 96-97 ]: ]: Clique is NP-hard to approximate to Clique is NP-hard to approximate to within n

within n ^{1- 1- } ; Optimal gap for 3-SAT and for Linear ; Optimal gap for 3-SAT and for Linear equations.

equations.

• Using Fourier analysis of the “marvelous” Long- Using Fourier analysis of the “marvelous” Long- Code Code , and a , and a Stronger PCP [Raz Stronger PCP [Raz ^{‘95 ‘95} ] ]

Hardness factor for VertexCover :

Hardness factor for VertexCover : 1.166 1.166 , for Max- , for Max-

CUT: CUT: 1.062 1.062

My Work My Work

• The Biased Long-Code: a generalization of the Long-Code. The Biased Long-Code: a generalization of the Long-Code.

[D.,Safra

[D.,Safra `02 `02 ] ]

• New perspectives on the Long Code yielding powerful new New perspectives on the Long Code yielding powerful new techniques.

techniques.

• Analysis of influence of variables on Boolean Functions Analysis of influence of variables on Boolean Functions

• Extremal Set Combinatorics. Extremal Set Combinatorics.

• New stronger enhancements of the PCP theorem, New stronger enhancements of the PCP theorem, [e.g. the [e.g. the Layered PCP in DGKR ’02, DRS ‘02]

Layered PCP in DGKR ’02, DRS ‘02]

• Leading to best-known inapproximability results for Leading to best-known inapproximability results for

• Vertex-Cover Vertex-Cover Hardness for vertex cover Hardness for vertex cover 1.367. 1.367 . [D.,Safra [D.,Safra `02 `02 ] ]

• Approximate Hypergraph Coloring Approximate Hypergraph Coloring Approximate hypergraph Approximate hypergraph coloring.

coloring. [D.,Regev,Smyth [D.,Regev,Smyth `02 `02 ] ]

• Hypergraph Vertex Cover Hypergraph Vertex Cover [D.; D.,Guruswami,Khot; [D.; D.,Guruswami,Khot;

D.,Guruswami,Khot,Regev

D.,Guruswami,Khot,Regev `02 `02 ] ]

• Proving hardness for gap-VC, we translate Proving hardness for gap-VC, we translate   into G and into G and then prove 2 things:

then prove 2 things:

I. I.

II. II.

• The hard part of the proof is part The hard part of the proof is part II, showing that II , showing that Every VC in G of size < (1+c)k can be

Every VC in G of size < (1+c)k can be decoded decoded into a into a satisfying assignment for

satisfying assignment for   . .

• In standard coding theory, we encode n bits by m bits (m>n), and are In standard coding theory, we encode n bits by m bits (m>n), and are able to recover “somewhat corrupt” codewords.

able to recover “somewhat corrupt” codewords.

“Every word, if close enough, we can decode” “ Every word, if close enough, we can decode”

• In In combinatorial decoding combinatorial decoding , we encode an assignment for , we encode an assignment for   by a by a

vertex cover in G and are able to recover “somewhat corrupt” vertex vertex cover in G and are able to recover “somewhat corrupt” vertex

covers.

covers.

“Every VC, if small enough, we can decode” “ Every VC, if small enough, we can decode”

VC(G) > (1+c)k VC(G) > (1+c)k

  is SAT is SAT VC(G)=k VC(G)=k

  is < 99% SAT is < 99% SAT

• Starting Point: the PCP theorem Starting Point: the PCP theorem

1. 1. Enhance it Enhance it

2. 2. Apply the Long-Code on small sub-components. Apply the Long-Code on small sub-components.

• The hardest part of these works is the The hardest part of these works is the interplay combining these two parts

interplay combining these two parts

PCP PCP ^{Enhanced Enhanced}

PCP PCP Long-Code Long-Code

The Underlying Structure

The Underlying Structure

Vertex Cover Vertex Cover

• A very loose outline of the construction: A very loose outline of the construction:

• A satisfying assignment can be encoded A satisfying assignment can be encoded into a vertex-cover. into a vertex-cover.

• A vertex-cover for the graph is a vertex cover in each H A vertex-cover for the graph is a vertex cover in each H . .

• Decode Decode each small vertex cover in H each small vertex cover in H into a value for the underlying into a value for the underlying y variable.

y variable.

• Then, show consistency between these values. Then, show consistency between these values.

• Combinatorial Question: Combinatorial Question: Construct such a graph H . Construct such a graph H .

  = = (y (y

v v y y

v v y y

) (y ) (y

v v y y

v v y y

) ( ) ( y y

v v y y

v v y y

) … (y ) … ( y

v v y y

v v y y

) )

y y _{1 1} y y _{2 2} y y _{3 3} y y _{m m}

Sub-Goal:

Sub-Goal: Construct a graph H Construct a graph H

Such that, Such that,

1. 1. Each Each value in {1,2,..,R} value in {1,2,..,R} corresponds corresponds to a small to a small vertex cover for

vertex cover for H H (i.e. of size k ~ ½ (i.e. of size k ~ ½ ¢ ¢ V V ). ).

2. 2. Every Every vertex cover for H vertex cover for H , if smaller than , if smaller than (2- (2-   ) ) ¢ ¢ k k

roughly corresponds

roughly corresponds to a single value in {1,2,..,R} to a single value in {1,2,..,R} . .

• Technique: Technique:

• Biased Long-Code, Biased Long-Code,

• Analysis of influence of variables on Boolean functions, Analysis of influence of variables on Boolean functions,

• Erdös-Ko-Rado theorems on intersecting families of subsets. Erdös-Ko-Rado theorems on intersecting families of subsets.

Long-Code of Long-Code of R R

 R elements, can be most concisely R elements, can be most concisely encoded by log R bits. encoded by log R bits.

 Seeking redundancy properties: we use Seeking redundancy properties: we use many more bits in the encoding. many more bits in the encoding.

 The Long-Code is the most redundant The Long-Code is the most redundant

way, using 2 way, using 2 ^{R R} bits. bits.

• One bit for every subset of [R] One bit for every subset of [R]

Long-Code of

Long-Code of R, R, ^{LC:[R] LC:[R]   {0,1} {0,1} 2 2 R R}

1 1 2 2 . . . . . . R R

• One bit for every subset of [R] One bit for every subset of [R]

• How do we encode the element i How do we encode the element i   [R]? [R]?

(What’s the value of LC(i)?) (What’s the value of LC(i)?)

Long-Code of

Long-Code of R, R, ^{LC:[R] LC:[R]   {0,1} {0,1} 2 2 R R}

0 0 1 1 1

1 1 2 2 . . . . . . R R

• Endow the bits with the product distribution: Endow the bits with the product distribution:

For each subset

For each subset F F , ,   _{p p} (F ( F ) = p ) = p ^{|F | F| |} (1-p) (1-p) ^{|R\F |R\ F| |}

• Roughly: take only subsets whose size is p Roughly: take only subsets whose size is p   R. R.

The p-

The p- Biased Biased Long-Code Long-Code

Biased Long-Code

Biased Long-Code

What is a codeword?

What is a codeword?

• A codeword is a vertex cover A codeword is a vertex cover

• The complement of a vertex-cover is always an independent set. The complement of a vertex-cover is always an independent set.

• In this graph, an independent set is an intersecting family of subsets. In this graph, an independent set is an intersecting family of subsets.

• Claim: a long-code codeword, i.e. {all subsets containing i} is a largest Claim: a long-code codeword, i.e. {all subsets containing i} is a largest independent set independent set , , and its complement, a

and its complement, a smallest smallest vertex cover vertex cover . .

• Maximal Intersecting Families of Subsets: Erdös-Ko-Rado ’61 Maximal Intersecting Families of Subsets: Erdös-Ko-Rado ’61

• Lemma: The Lemma: The  

size of an intersecting family is  size of an intersecting family is  p (proof using “shadows” [Kruskal `63, p (proof using “shadows” [Kruskal `63, Katona `68])

Katona `68])

• Much more difficult to prove: Any vertex cover whose size is < 1-p Much more difficult to prove: Any vertex cover whose size is < 1-p

is “decodable” is “decodable” into a into a value in 1,…,R.

value in 1,…,R. (combinatorial decoding) (combinatorial decoding)

• Using: the complete characterization of maximal intersecting families by Ahlswede and Using: the complete characterization of maximal intersecting families by Ahlswede and Khachatrian ’97

Khachatrian ’97, and , and Friedgut’s Theorem on when Boolean Functions are Juntas, etc. Friedgut’s Theorem on when Boolean Functions are Juntas, etc.

VC(G)=1-p = 2/3

VC(G)=1-p = 2/3 VC(G) < (1-p

) = 8/9

We constructed a graph s.t., We constructed a graph s.t.,

1. 1. Each Each value in {1,2,..,R} value in {1,2,..,R} corresponds corresponds to a small vertex to a small vertex cover for

cover for H H (i.e. of size k (i.e. of size k ). ).

2. 2. Every Every vertex cover for H vertex cover for H , if smaller than , if smaller than (4/3) (4/3) ¢k ¢ k roughly roughly corresponds

corresponds to a single value in {1,2,..,R} to a single value in {1,2,..,R} . .

Now we can plug it into the whole construction…

Now we can plug it into the whole construction…

(2- (2-

  )k )k ??? ???

VC(G)=1-p = 2/3

VC(G)=1-p = 2/3 VC(G) < (1-p

) = 8/9

Future Directions Future Directions

• Finding the true threshold: (stronger Finding the true threshold: (stronger

“combinatorial decoding”)

“combinatorial decoding”)

• Factor 2 inapproximability for Vertex Cover Factor 2 inapproximability for Vertex Cover

• Other problems: approximate c Other problems: approximate c o o l l o o r r i i n n g g , etc. , etc.

• Simplification of PCP, locally testable codes. Simplification of PCP, locally testable codes.

• Decoding in completely different contexts, with Decoding in completely different contexts, with applications for database privacy.

applications for database privacy.

Thanks

Thanks