Sept 17, Undergrad COMPLEXITY TURING MACHINES AND P NON-DETERMINISTIC TURING MACHINES AND NP NP-COMPLETENESS: THE COOK-LEVIN THEOREM

15-455

Undergrad COMPLEXITY

Sept 17, 2013

NON-DETERMINISTIC

TURING MACHINES AND NP

TURING MACHINES AND P

NP-COMPLETENESS:

THE COOK-LEVIN THEOREM

TURING MACHINES AND NP

TURING MACHINE

read write move

→ , R q_accept 0→ 0, R → , R q_reject 0→ 0, R → , L 0 → 0, R

read write move

→ , R q_accept 0→ 0, R → , R 0→ 0, R → , L

Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, , q₀, q_accept, q_reject), where:

Q is a finite set of states q₀ Q is the start state

q_accept Q is the accept state

Γ is the tape alphabet, where  Γ and Σ  Γ

Σ is the input alphabet, where  Σ

 : Q’  Γ → Q  Γ  {L, R}

q_reject Q is the reject state, and q_reject q_accept

We can encode a TM as a finite string of 0s and 1s

0

₁₀

₁₀

₁₀

₁₀

₁₀

₁₀

_1”

_”

n states

start

state reject state

m tape symbols (first k are input symbols) accept state blank symbol

( (p, a), (q, b, L) ) = 0

₁₀

₁₀

₁₀

₁₀

( (p, a), (q, b, R) ) = 0

₁₀

₁₀

₁₀

₁₁

CONFIGURATIONS

11010q

₇

00110

A Turing Machine M accepts input w if there is a sequence of configurations C₁, … , C_k such that

1. C₁is a start configuration of M on input w, ie C₁is q₀w

2. each C_iiyieldsy C_i+1i+1, ie M can legally go from , g y g C_ito C_i+1 in a single step

ua q_ibv yields u q_jacv if  (qi, b) = (qj, c, L)

ua qi bv yields uac q_jv if  (qi, b) = (qj, c, R)

A Turing Machine M accepts input w if there is a sequence of configurations C₁, … , C_k such that

1. C₁is a start configuration of M on input w, ie C₁is q₀w

2. each C_iiyieldsy C_i+1i+1, ie M can legally go from , g y g C_ito C_i+1 in a single step

3. C_kis an accepting configuration, ie the state of the configuration is q_accept

C₁, … , C_k is called an accepting computation history of M on w.

A TM recognizes a language L  Σ* iff it accepts all and only those strings in the language

A language L  Σ* is called

Turing-recognizable or recursively enumerable or

semi-decidableiff some TM recognizes L

A TM decides a language L  Σ* iff it accepts

all strings in L and rejects all strings not in L A language L  Σ* is called decidable or

recursive iff some TM decides L

A language is called Turing-recognizable or semi-decidable or recursively enumerable (r.e.) if some TM recognizes it

A language is called decidable or recursive if some TM decides it decidable (recursive) languages semi-decidable (r.e.) languages Languages over {0,1}

A language is called Turing-recognizable or semi-decidable or recursively enumerable (r.e.) if some TM recognizes it

A language is called decidable or recursive if some TM decides it

Theorem. A Language is decidable if and

only if both it and it’s complement are

semi-decidable.

A language is called Turing-recognizable or semi-decidable or recursively enumerable (r.e.) if some TM recognizes it

A language is called decidable or recursive if some TM decides it

A = { (M w) | M is a TM that accepts string w } A_TM= { (M,w) | M is a TM that accepts string w } HALT_TM= { (M,w) | M is a TM that halts on string w }

Theorem. Both are examples of

semi-decidable, undecidable languages.

THE CHURCH-TURING THESIS

Intuitive Notion of Algorithms

EQUALS

Turing Machines

ORACLE TMs

Each query answered in one step

We say A is decidable in B (A Turing reduces to B) if there is an oracle TM M with oracle B

that decides A

A Turing Reduces to B

A



_T

B

A Mapping Reduces to B

Machine

Many-One

A



_m

B

A _f B

Suppose f : Σ*  Σ* is a computable function

such that w  A  f(w)  B

Σ*

Σ*

A_mB f

Say: A is Mapping Reducible to B

Write: A



B

A _f B

Σ*

Σ*

A_mB

Suppose f : Σ*  Σ* is a computable function

such that w  A  f(w)  B

f

So, if B is (semi) decidable, then so is A (And if  B is (semi) decidable, then so is  A)

A_TM HALTTM

f

Σ*

Σ*

A_TM= { (M,w) | M is a TM that accepts string w } HALT_TM= { (M,w) | M is a TM that halts on string w }

A_TMmHALTTM f f: (M,w)  (M’, w) whereM’(s) = M(s) ifM(s) accepts, Loops otherwise So, (M, w) ATM (M’, w)  HALTTM s  Σ*



VERSUS 

If A mB then there is a computable function

f : Σ*  Σ*, where for every w,

A f( ) B w  A  f(w)  B

We can thus use an oracle for B to decide A

Theorem:HALTTMTHALTTM

But, Theorem:HALT_TM_mHALT_TM

_WHY?

COMPLEXITY THEORY

Studies what can and can’t be computed under limited resources such as time, space, etc

Today: Time complexity

MEASURING TIME COMPLEXITY

1. Scan across the tape and reject if the ~n

On input of length n:

string is not of the form 0i₁j

2. Repeat the following if both 0s and 1s remain on the tape:

Scan across the tape, crossing off a single 0 and a single 1

3. If 0s remain after all 1s have been crossed off, or vice-versa, reject. Otherwise accept. ~n

~n2

Definition: Let M be a TM that halts on all

inputs.

The running time or time-complexity of M

g

p

y

is the function f : N

 N, where f(n) is the

maximum number of steps that M uses on

any input of length n.

Definition: TIME(t(n)) = { L | L is a language decided by a O(t(n)) time Turing Machine }

A = { 0k₁k_{| k  0 }  TIME(n}2₎

A = { 0k₁k_{| k  0 }  TIME(n logn)}

A = { 0

₁

_{| k}

_{ 0 }  TIME(nlog n)}

Cross off every other 0 and every other 1. If the total # of 0s and 1s left on the tape is odd, reject

00000000000001111111111111

0 0 0 0 0 0 1 1 1 1 1 1

x0x0x0x0x0x0xx1x1x1x1x1x1x

xxx0xxx0xxx0xxxx1xxx1xxx1x

xxxxxxx0xxxxxxxxxxxx1xxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxx

Can A = { 0k₁k_{| k  0 } be decided in time O(n)}

ith t t TM?

We can prove that a TM cannot decide

this A in less time than O(nlog n)

Scan all 0s and copy them to the second tape. Scan all 1s, crossing off a 0 from the second tape for each 1.

Different models of computation

yield different running times for

the same language!

the same language!

Poly-time conversion

Theorem: Let t(n) be a function such that t(n)  n.

Then every t(n)-time multi-tape TM has an equivalent O(t(n)2_{) single tape TM}

Claim: Simulating each step in the multi-Claim: Simulating each step in the multi tape machine uses at most O(t(n)) steps on a single-tape machine.

P = TIME(n



k

₎

k

N

k

 N

NON-DETERMINISTIC

TURING MACHINES AND NP

0 → 0, R

read write move

q_accept → , R 0 → 0, R → , R q_reject 0→ 0, R 0 → 0, R

read write move

q_accept → , R 0 → 0, R q_reject 0→ 0, R

NON-DETERMINISTIC TMs

1. The machine may proceed according to several possibilities

2. The machine accepts a string if there exists a path from start configuration to an accepting configuration

Definition: A Non-Deterministic TM is a 7-tuple T = (Q, Σ, Γ, , q₀, q_accept, q_reject), where:

Q is a finite set of states

Γ is the tape alphabet, where  Γ and Σ  Γ

Σ is the input alphabet, where  Σ

q₀ Q is the start state  : Q  Γ → 2(Q  Γ  {L,R})

q_accept Q is the accept state

Deterministic Computation

Non-Deterministic Computation

j t

accept or reject accept reject

The non-deterministic machine accepts a string if there exists a

path from start configuration to an accepting configuration

Definition: Let M be a NTM that halts on

all inputs.

The running time or time-complexity of M

is the function f : N

N where f(n) is the

is the function f : N

 N, where f(n) is the

maximum number of steps that M uses on

any branch of its computation on any

input of length n.

accept or reject accept reject

Theorem: Let t(n) be a function such that t(n)  n. Then

every t(n)-time nondeterministic single-tape TM has an equivalent 2O(t(n)) _{deterministic single tape TM}

{ L | L is decided by a O(t(n))-time

non-deterministic Turing machine }

Definition: NTIME(t(n))

TIME(t(n))

 NTIME(t(n))

BOOLEAN FORMULAS

(

x  y)  z

 =

A satisfying assignment is a setting of the variables that makes the formula true

x = 1, y = 1, z = 1 is a satisfying assignment for 

A Boolean formula is satisfiable if there exists a satisfying assignment for it

(

)

a

 b  c  d

YES

SAT = {  |  is a satisfiable Boolean formula }

(x  y)  x

A 3cnf-formula is of the form: (x₁ x2 x3)  (x4 x2 x5) (x3 x2 x1) clauses (x₁ x2 x1) YES literals ( _{1 2 1}) (x₃ x1) (x3 x2 x1) (x₁ x₂ x₃)  (x₄ x₂ x₁) (x₃ x₁ x₁) (x₁ x2 x3) (x3 x2 x1) NO NO NO

3SAT = {  |  is a satisfiable 3cnf-formula }

Theorem: SAT, 3SAT  NTIME(n2₎

1. Check if the formula is boolean, or in 3cnf On input :

2. For each variable, non-deterministically substitute it with 0 or 1

3. Test if the assignment satisfies 

( x  y x ) ( 0   y  0 ) ( 1   y  1 ) ( 0   0  0 ) ( 0   1  0 )

NP = NTIME(n



k

₎

k

N

k

 N

Theorem: L  NP  if there exists a poly-time

Turing machine V(x,y) (verifier) with

L = { x | y(witness) |y| = poly(|x|) and V(x,y) accepts } Proof:

<=) If L = { x | y |y| = poly(|x|) and V(x,y) accepts }

then L  NP:

Because we can guess y and then run V =>) If L  NP then

L = { x | y |y| = poly(|x|) and V(x,y) accepts }:

Let N be a non-deterministic poly-time TM that decides L and define V(x,y) to accept if y is an accepting computation history of N on x

A language is in NP if and only if there exist

polynomial-time certificates for

membership to the language

SAT

{

 | y such that y is a satisfying

assignment to

g

 }

 }

SAT is in NP because a satisfying

assignment is a polynomial-time certificate

that a formula is satisfiable

CLIQUE = { (G,k) | G is an undirected

graph with a k-clique }

Theorem: CLIQUE

 NP

The k-clique itself is a certificate

b a e c d f g

NP = all the problems for which

once you have the answer it is

y

easy (i.e. efficient) to verify

P = NP?

$$$

POLY-TIME REDUCIBILITY

function

Language A is polynomial time reducible to language B written A B if there is a poly

if some poly-time Turing machine M, on every input w, halts with just f(w) on its tape language B, written A _PB, if there is a poly-time computable function f : Σ*  Σ* such that:

w  A  f(w)  B

f is called a polynomial time reduction of A to B

A B

f

Σ* Σ*

f

Theorem: If A PB and B  P, then A  P

Theorem: If A PB and B  P, then A  P

Proof: Let M_Bbe a poly-time (deterministic) TM that decides B and let f be a poly-time reduction from A to B

We build a machine M_Athat decides A as follows: On input w:

1. Compute f(w) 2. Run M_Bon f(w)

Definition: A language B is NP-complete if: 1. B  NP (i.e. B is NP-easy)

2. Every A in NP is poly-time reducible to B (i.e. B is NP-hard)

Suppose B is NP-Complete

P

NP

B

So, if B is NP-Complete and B  P then NP = P.

NP-COMPLETENESS:

THE COOK-LEVIN THEOREM

Theorem

: SAT is NP-complete

Corollary: SAT

 P if and only if P = NP

Steve Cook Leonid Levin

Theorem (Cook-Levin): SAT is NP-complete Proof:

(1) SAT  NP

(2) Every language A in NP is polynomial time reducible to SAT

f S

We build a poly-time reduction from A to SAT

Let N be a non-deterministic TM that decides A in time nk

The reduction turns a string w into a 3-cnf formula  such that w  A iff   3-SAT.

 will simulate the NP machine N for A on w.

A 3SAT

f

w



f

The reduction f turns a string w into a 3-cnf formula 

such that: w  A    3SAT.

 will “simulate” the NP machine N for A on w.

Deterministic Computation Non-Deterministic Computation j t nk

accept or reject accept reject

exp(nk₎

Suppose A  NTIME(nk_{) and let N be an NP machine for A.}

A tableau for N on w is an nk_{ n}k_{table whose rows are the}

configurations of some possible computation of N on input w.

q0 w1w2 wn # … …  # # # # # nk nk

A tableau is accepting if any row of the tableau is an accepting configuration

Determining whether N accepts w is equivalent to determining whether there is an accepting tableau for N on w

Given w, our 3cnf-formula  will describe a

generic tableau for N on w (in fact, essentially genericfor N on any string w of length n). The 3cnf formula  will be satisfiable if and only

ifthere is an accepting tableau for N on w.

Let C = Q  Γ  { # }

For each i and j (1  i, j  nk_{) and for each s C}

VARIABLES of 

Each of the (nk₎2_{entries of a tableau is a cell}

cell[i,j] = the cell at row i and column j j ( , j )

we have a variable x_i,j,s

These are the variables of  and represent the

contents of the cells

We will have: x_i,j,s= 1  cell[i,j] = s

# variables = |C|n2k_{, ie O(n}2k_{), since |C| only depends on N}

x

_i,j,s

= 1

means

cell[ i, j ] = s

We now design  so that a satisfying assignment

to the variables x_i,j,scorresponds to an accepting tableau for N on w

The formula  will be the AND of four parts:

 = 

 

 

 

cellensures that for each i,j, exactly one xi,j,s= 1

acceptensures* that an accepting configuration occurs

somewhere in the table

startensures that the first row of the table is the starting

(initial)configuration of N on w

moveensures* that every row is a configuration that legally

 C

x_i,j,s (xi,j,s xi,j,t) cell =

t C

_cellensures that for each i,j, exactly one x_i,j,s= 1 1  i, j  nk s  C s,t  C s ≠ t at least one variable is turned on at most one variable is turned on



= x

 x



x

 x

 …  x



x

 …  x

 x



=

x



configuration occurs somewhere in the table



It works by ensuring that each 2  3 “window”

of cells is legal (does not violate N’s rules)

q0 w1 w2 wn

# … …  #

# #

# #

If (q1,a) = {(q1,b,R)} and (q1,b) = {(q2,c,L), (q2,a,R)}

Which of the following windows are legal: a q₁ b q₂ a c a a q₁ a a b a q₁ b q₁ a a # b a # b a a b a a a a a q₁ b a a q₂ a b a a b q₂ b b b c b b b q₁ b q₂ b q₂ CLAIM: If

• the top row of the table is the start configuration,

and

• and every window is legal,

Then

each row of the table is a configuration that legally follo s the preceding one a q

follows the preceding one. Proof:

Inupper configuration, every cell that doesn’t contain the

boundary symbol #, is the center top cell of a window. Case 1. center cell of window is a non-state symbol and not adjacent to a state symbol

Case 2. center cell of window is a state symbol

a q

CLAIM: If

• the top row of the table is the start configuration,

and

• and every window is legal,

Then

each row of the table is a configuration that legally follo s the preceding one a q

follows the preceding one. a q a

Proof:

Inupper configuration, every cell that doesn’t contain the

boundary symbol #, is the center top cell of a window.

So the lower configuration follows from the upper!!!

q0 w1 w2 wn  # … …  # # ok # o k k ok w3 w4 w2 w3 w4 # # Proof:

Inupper configuration, every cell that doesn’t contain the

boundary symbol #, is the center top cell of a window.

So the lower configuration follows from the upper!!!

a1 q a2 an  # … …  # # ok ok # a3 a4 a3 a4 ok a5 a5 # # o k k Proof:

Inupper configuration, every cell that doesn’t contain the

boundary symbol #, is the center top cell of a window.

So the lower configuration follows from the upper!!!

a1 q a2 an  # … …  # # ok ok # a3 a4 a3 a4 ok a5 a5 # # o k k Proof:

Inupper configuration, every cell that doesn’t contain the

boundary symbol #, is the center top cell of a window.

So the lower configuration follows from the upper!!!

(i,j-1) a₁ (i,j) a₂ (i,j+1) a₃ row i

col. j-1 col. j col. j+1

(i+1,j-1) a₄ (i+1,j) a₅ (i+1,j+ 1) a₆ row i+1

The (i,j) Window



=

1  i, j  nk

( the (i, j) window is legal )

the (i, j) window is legal =

a1, …, a6

is a legal window

( xi,j-1,a₁ xi,j,a₂ xi,j,+1,a₃  xi+1,j-1,a  xi+1,j,a  xi+1,j+1,a )

4 5 6

g

This is a disjunct over all (≤ |C|6_{) legal sequences (a} 1, …, a6).



=

1  i, j  nk

( the (i, j) window is legal )

the (i, j) window is legal =

a1, …, a6

is a legal window

( xi,j-1,a₁ xi,j,a₂ xi,j,+1,a₃  xi+1,j-1,a  xi+1,j,a  xi+1,j+1,a )

4 5 6

g

This disjunct is satisfiable



There is some assignment to the cells (ie variables) in the window (i,j) that makes the window legal

This is a disjunct over all (≤ |C|6_{) legal sequences (a} 1, …, a6).



=

1  i, j  nk

( the (i, j) window is legal )

the (i, j) window is legal =

a1, …, a6

is a legal window

( xi,j-1,a₁ xi,j,a₂ xi,j,+1,a₃  xi+1,j-1,a  xi+1,j,a  xi+1,j+1,a )

4 5 6

g

So move is satisfiable 

There is some assignment to each of the variables that makes every window legal.

This is a disjunct over all (≤ |C|6_{) legal sequences (a} 1, …, a6).



=

1  i, j  nk

( the (i, j) window is legal )

the (i, j) window is legal =

a1, …, a6

is a legal window

( xi,j-1,a₁ xi,j,a₂ xi,j,+1,a₃  xi+1,j-1,a  xi+1,j,a  xi+1,j+1,a )

4 5 6

g

This is a disjunct over all (≤ |C|6_{) legal sequences (a} 1, …, a6). Can re-write as equivalent conjunct over all illegal windows:

a1, …, a6

ISN’T a legal window

( x

-

- -

-

-

-

the (i, j) window is legal =

 is satisfiable (ie, there is some assignment to each of the varialbes s.t. evaluates to 1)



there is some assignment to each of the variables s.t. cell andstart and acceptandmove each evaluates to 1



 = 

 

 

 



There is some assignment of symbols to cells in the tableau such that:

• The first row of the tableau is a start configuration and • Every row of the tableau is a configuration that follows from the

preceding by the rules of N and • One row is an accepting configuration



There is some accepting computation for N with input w

WHAT’S THE

LENGTH OF ?

 = 

 

 

 

 1  i, j  nk s  C

x_i,j,s (x_i,j,s x_i,j,t)



s,t  C

s ≠ t

 = 

 

 

 

O(n

2k

_{) clauses}

Length(cell ) = O(n2k) O(log (n)) = O(n2klog n) length(indices)

 = 

 

 

 



= x

 x



x

 x

 …  x



x

 …  x

 x

O(n

k

₎



=

x

 = 

 

 

 

O(n

2k

₎



=

1  i, j  nk

( the (i, j) window is legal )

( xi,j-1,a₁ xi,j,a₂ xi,j,+1,a₃  xi+1,j-1,a₄  xi+1,j,a₅  xi+1,j+1,a) 6

-

- -

-

-

-the (i, j) window is legal =

O(n

2k

₎

a1, …, a6

ISN’T a legal window

1 2 3 4 5 6

This is a conjunct over all (≤ |C|6_{) illegal sequences (a} 1, …, a6).

Theorem (Cook-Levin): SAT is NP-complete

Corollary: SAT

 P if and only if P = NP

Theorem (Cook-Levin): 3SAT is NP-complete

Corollary: 3SAT

 P if and only if P = NP

3-SAT?

How do we convert the whole thing into a 3-cnf formula?

Everything was an AND of ORs

We just need to make those ORs with 3 literals

If l h l th th i bl

a ≡ (a  a  a), (a  b) ≡ (a  b  b)

If a clause has less than three variables:

(a  b  c  d) ≡(a  b  z)  (z  c  d)

If a clause has more than three variables: (a₁ a₂ …  a_t) ≡

A 3SAT f

w



f

Given A in NP. The reduction f turned a string w into a 3-cnf formula





A 3SAT

f

w



f

The reduction f is poly time.

WHY?



3-SAT is NP-Complete

P

NP

A

3-SAT

Theorem (Cook-Levin): 3SAT is NP-complete

Corollary: 3SAT

 P if and only if P = NP