Halting, Undecidability, and Maybe Some Complexity

(1)

Algorithms & Models of Computation

CS/ECE 374, Fall 2020

Halting, Undecidability, and Maybe

Some Complexity

Lecture 9

Tuesday, September 22, 2020

(2)

Quote

“Young man, in mathematics you don’t understand things. You just get used to them.” – John von Neumann.

(3)

Algorithms & Models of Computation

9.1 Cantor’s diagonalization argument

(4)

You can not count the real numbers

I = (0, 1).

N = {1, 2, 3, . . .}the integer numbers

Claim (Cantor)

|N| 6= |I |

Claim (Warm-up)

|N| ≤ |I |

(5)

You can not count the real numbers

I = (0, 1).

N = {1, 2, 3, . . .}the integer numbers

Claim (Cantor)

|N| 6= |I |

Claim (Warm-up)

|N| ≤ |I |

Proof.

|N| ≤ |I | exists a one-to-one mapping from_N to I. One such mapping isf (i ) = 1/i, which readily implies the claim.

(6)

You can not count the real numbers II

I = (0, 1), _{N = {1, 2, 3, . . .}}.

Claim (Cantor)

|N| 6= |I |, where I = (0, 1).

Proof.

Write every number in(0, 1) in its decimal expansion. E.g., 1/3 = 0.33333333333333333333 . . ..

Assume that_{|N| = |I |}. Then there exists a one-to-one mapping _{f : N → I}. Let βi be

(7)

You can not count the real numbers II

I = (0, 1), _{N = {1, 2, 3, . . .}}.

Claim (Cantor)

|N| 6= |I |, where I = (0, 1).

Proof.

Write every number in(0, 1) in its decimal expansion. E.g.,

1/3 = 0.33333333333333333333 . . ..

Assume that_{|N| = |I |}. Then there exists a one-to-one mapping _{f : N → I}. Let βi be

the ith digit of f (i ) ∈ (0, 1).

di = any number in {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} \ {di −1, βi} D = 0.d1d2d3. . . ∈ (0, 1).

(8)

The matrix...

f (1) f (2) f (3) f (4) . . . 1 1 1 0 0 . . . 2 0 1 0 1 . . . 3 1 0 1 1 . . . 4 0 1 0 0 . . . .. . ... ... ... ... . . .

(9)

The matrix...

f (1) f (2) f (3) f (4) . . . 1 β1 = 1 1 0 0 . . . 2 0 β2 = 1 0 1 . . . 3 1 0 β3 = 1 1 . . . 4 0 1 0 β4 = 0 . . . .. . ... ... ... ... . . .

(10)

The matrix...

f (1) f (2) f (3) f (4) . . . 1 1 1 0 0 . . . 2 0 1 0 1 . . . 3 1 0 1 1 . . . 4 0 1 0 0 . . . .. . ... ... ... ... . . .

di = any number in {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} \ {di −1, βi} =⇒ ∀i βi 6= di.

(11)

The matrix...

f (1) f (2) f (3) f (4) . . . 1 1 1 0 0 . . . 2 0 1 0 1 . . . 3 1 0 1 1 . . . 4 0 1 0 0 . . . .. . ... ... ... ... . . .

D = 0.23232323 . . ..

(12)

The matrix...

f (1) f (2) f (3) f (4) . . . 1 1 1 0 0 . . . 2 0 1 0 1 . . . 3 1 0 1 1 . . . 4 0 1 0 0 . . . .. . ... ... ... ... . . .

(13)

The liar paradox

When one day an expedition was sent to the spatial coordinates that Voojagig had claimed for this planet they discovered only a small asteroid inhabited by a solitary old man who claimed repeatedly that nothing was true, though he was later discovered to be lying.

– The Hitchhiker Guide to the Galaxy

1 The liar’s paradox: This sentence is false. 2 Related to Russell’s paradox.

3 Omnipotence paradox: Can [an omnipotent being] create a stone so heavy that it

(14)

The liar paradox

(15)

The liar paradox

(16)

The liar paradox

(17)

THE END

...

(18)

Algorithms & Models of Computation

9.2 Introduction to the halting theorem

(19)

The halting problem

Halting problem:

Given a program Q, if we run it would it stop?

Q:

Can one build a program P, that always stops, and solves the halting problem.

Theorem (“Halting theorem”)

(20)

The halting problem

Halting problem:

Given a program Q, if we run it would it stop?

Q:

Can one build a program P, that always stops, and solves the halting problem.

Theorem (“Halting theorem”)

(21)

Intuition, why solving the Halting problem is really hard

Definition

An integer number n is a weird number if

the sum of the proper divisors (including 1 but not itself) ofn the number is > n, no subset of those divisors sums to the number itself.

70 is weird. Its divisors are 1, 2, 5, 7, 10, 14, 35. 1 + 2 + 5 + 7 + 10 + 14 + 35 = 74. No subset of them adds up to 70.

Open question:

Are there are any odd weird numbers?

Write a programP that tries all odd numbers in order, and check if they are weird. The programs stops if it found such number.

(22)

Intuition, why solving the Halting problem is really hard

Definition

Open question:

(23)

Intuition, why solving the Halting problem is really hard

Definition

Open question:

(24)

Intuition, why solving the Halting problem is really hard

Definition

Open question:

(25)

If you can halt, you can prove or disprove anything...

1 Consider any math claimC. 2 Prover algorithmPC:

(A) Generate sequence of all possible proofs (sequence of strings) into a pipe/queue. (B) hpi ←pop top of queue.

(C) FeedhpiandhC i, into a proof verifier (“easy”). (D) Ifhpivalid proof ofhC i, then stop and accept.

(E) Go to (B).

3 P

C halts ⇐⇒ C is true and has a proof.

(26)

If you can halt, you can prove or disprove anything...

(A) Generate sequence of all possible proofs (sequence of strings) into a pipe/queue.

(B) hpi ←pop top of queue.

(E) Go to (B).

3 P

(27)

If you can halt, you can prove or disprove anything...

(E) Go to (B).

3 P

(28)

If you can halt, you can prove or disprove anything...

(C) FeedhpiandhC i, into a proof verifier (“easy”).

(D) Ifhpivalid proof ofhC i, then stop and accept.

(E) Go to (B).

3 PC halts ⇐⇒ C is true and has a proof.

(29)

THE END

...

(30)

Algorithms & Models of Computation

9.3 The halting theorem

(31)

Encodings

M: Turing machine

hMi: a binary string uniquely describing M (i.e., it is a number.

w: An input string.

hM, w i: A unique binary string encoding bothM and input w. A_TM = nhM, w i

M is a TM and M accepts w o

(32)

Encodings

M: Turing machine

w: An input string.

hM, w i: A unique binary string encoding bothM and input w.

A_TM = nhM, w i

(33)

Encodings

M: Turing machine

w: An input string.

hM, w i: A unique binary string encoding bothM and input w.

A_TM = nhM, w i

(34)

Complexity classes

Regular

Context free grammar Turing decidable Turing recognizable Not Turing recognizable.

(35)

A

TM

is TM recognizable...

A_TM = nhM, w i M is a TM and M accepts w o .

Lemma

A_TM is Turing recognizable.

Proof.

Input: hM, w i.

UsingUTM simulate running M onw. If M acceptsw then accept, if M rejects then reject. Otherwise, the simulation runs forever.

(36)

A

TM

is TM recognizable...

A_TM = nhM, w i M is a TM and M accepts w o .

Lemma

A_TM is Turing recognizable.

Proof.

Input: hM, w i.

(37)

A

TM

is not TM decidable!

A_TM =nhM, w i

.

Theorem (The halting theorem.)

A_TM is not Turing decidable.

Proof:

AssumeA_TM is_TM decidable...

Halt: TMdeciding ATM. Halt always halts, and works as follows:

HalthM, w i= (

accept M accepts w

(38)

A

TM

is not TM decidable!

A_TM =nhM, w i

.

Theorem (The halting theorem.)

Proof:

HalthM, w i= (

accept M accepts w

(39)

A

TM

is not TM decidable!

A_TM =nhM, w i

.

Theorem (The halting theorem.)

Proof:

HalthM, w i= (

accept M accepts w

(40)

Halting theorem proof continued 1

We build the following new function:

Flipper(hMi)

res ← Halt(hM, Mi) if resis accept then

reject

else

accept

Flipperalways stops:

FlipperhMi= (

(41)

Halting theorem proof continued 1

We build the following new function:

Flipper(hMi)

res ← Halt(hM, Mi) if resis accept then

reject

else

accept Flipperalways stops:

FlipperhMi= (

reject M accepts hMi

(42)

Halting theorem proof continued 2

FlipperhMi= (

accept M does not accept hMi .

Flipperis a TM (duh!), and as such it has an encoding hFlipperi. RunFlipper on

itself:

Flipper

hFlipperi= (

reject Flipperaccepts hFlipperi

accept Flipperdoes not accept hFlipperi . This is absurd. Ridiculous even!

(43)

Halting theorem proof continued 2

FlipperhMi= (

itself:

Flipper

hFlipperi= (

accept Flipperdoes not accept hFlipperi .

This is absurd. Ridiculous even!

(44)

Halting theorem proof continued 2

FlipperhMi= (

itself:

Flipper

hFlipperi= (

accept Flipperdoes not accept hFlipperi .

(45)

But where is the diagonalization argument????

hM1i hM2i hM3i hM4i . . .

M1 rej acc rej rej . . .

M2 rej acc rej acc . . .

M3 acc acc acc rej . . . M4 rej acc acc rej . . . ..

(46)

THE END

...

(47)

Algorithms & Models of Computation

9.4 Unrecognizable

(48)

TM recognizable

Definition

LanguageL is _TMdecidable if there exists M that always stops, such that L(M) = L.

Definition

LanguageL is _TMrecognizable if there exists M that stops on some inputs, such that L(M) = L.

Theorem (Halting)

A =nhM, w i M is a _{and M accepts w} o

(49)

TM recognizable

Definition

LanguageL is _TMrecognizable if there exists M that stops on some inputs, such that

L(M) = L.

Theorem (Halting)

A_TM =nhM, w i

. isTM recognizable, but not decidable.

(50)

TM recognizable

Definition

LanguageL is _TMrecognizable if there exists M that stops on some inputs, such that

L(M) = L.

Theorem (Halting)

A =nhM, w i M is a _{and M accepts w} o

(51)

TM recognizable

Lemma

If Land L = Σ∗ _{\ L} _{are both}

TMrecognizable, then L and L are decidable.

Proof.

M: _TM recognizing L. Mc: TMrecognizing L.

Given inputx, using UTM simulating running M and Mc on x in parallel. One of

them must stop and accept. Return result. =⇒ L is decidable.

(52)

TM recognizable

Lemma

If Land L = Σ∗ _{\ L} _{are both}

TMrecognizable, then L and L are decidable.

Proof.

M: _TM recognizing L.

Mc: TMrecognizing L.

Given inputx, using UTM simulating running M and Mc on x in parallel. One of

them must stop and accept. Return result.

(53)

Complement language for

A

TM

A_TM = Σ∗ \nhM, w i

.

But don’t really care about invalid inputs. So, really: A_TM =nhM, w i

M is aTM and M does not accept w o

(54)

Complement language for

A

TM

A_TM = Σ∗ \nhM, w i

.

But don’t really care about invalid inputs. So, really:

A_TM =nhM, w i

M is aTM and M does not accept w o

(55)

Complement language for

A

TM

is not TM-recognizable

Theorem

The language

A_TM =nhM, w i

M is a TM and M does not accept w o . is not_TM recognizable.

Proof.

A_TM is TM-recognizable. If A_TM is TM-recognizable =⇒ (by Lemma) A is decidable. A contradiction.

(56)

Complement language for

A

TM

is not TM-recognizable

Theorem

The language

A_TM =nhM, w i

Proof.

A_TM is TM-recognizable. If A is -recognizable

(57)

Complement language for

A

TM

is not TM-recognizable

Theorem

The language

A_TM =nhM, w i

Proof.

A_TM is TM-recognizable. If A_TM is TM-recognizable =⇒ (by Lemma) A is decidable. A contradiction.

(58)

THE END

...

(59)

Algorithms & Models of Computation

9.5 Turing complete

(60)

Equivalent to a program

Definition

A system isTuring complete if one can simulate a Turing machine using it.

1 Programming languages (yey!). 2 C++ templates system (boo). 3 John Conway’s game of life. 4 Many games (Minesweeper). 5 Post’s correspondence problem.

(61)

Equivalent to a program

Definition

A system isTuring complete if one can simulate a Turing machine using it.

1 Programming languages (yey!). 2 C++ templates system (boo). 3 John Conway’s game of life. 4 Many games (Minesweeper). 5 Post’s correspondence problem.

(62)

Post’s correspondence problem

S: set of domino tiles.

abb

bc : domino piece a string at the top and a string at the bottom.

Example: S = b ca , a ab , ca a , abc c .

(63)

Matching dominos

S = b ca , a ab , ca a , abc c .

matchfor S: ordered list of dominos fromS, such that top strings make same string as bottom strings. Example:

a ab b ca ca a a ab abc c .

(1) Can use same domino more than once. (2) Do not have to use all pieces ofS.

(64)

Matching dominos

a ab b ca ca a a ab abc c . (1) Can use same domino more than once.

(65)

Matching dominos

a ab b ca ca a a ab abc c .

(1) Can use same domino more than once. (2) Do not have to use all pieces ofS.

(66)

Post’s Correspondence Problem

Post’s Correspondence Problem (PCP) is deciding whether a set of dominos has a

match or not.

modified Post’s Correspondence Problem(_MPCP):_PCP + a special tile.

Matches forMPCP have to start with the special tile.

Theorem

(67)

THE END

...