IST 4 Information and Logic

(1)

IST 4

Information and Logic

(2)

mon tue wed thr fri

1 _M₁

1

8 _M₁

15

1 2

^M²

22 PCP

2

29 _M₂

6

3

13

3 4

20 PCP 4 5

27

3

5

x= hw#x out x= hw#x due

Mx= MQx out Mx= MQx due

oh oh

oh

oh oh

oh

oh oh oh

oh

oh oh

oh = office hours ^oh

T

= today

T oh

oh

sun

PCP= Programming Challenge

Midterms

(3)

- Lecture 1: Life: DNA sequences and evolution - Lecture 2: Human brain: natural languages

- Lecture 3: Artificial languages: numbers and writing

Information and Logic

- Lecture 4: Languages for quantities: Babylonian mathematics

- Lecture 5: Contrast: Babylonian mathematics vs Greek mathematics - Lecture 6: Flow: Euclid to Algorizmi to Fibonacci to Leibniz

- Lecture 9: Boolean Algebra – a new language for logic - Lecture 7: Beyond arithmetic: a language of syntax boxes - Lecture 8: Leibniz: syntax for reasoning, Aristotle and Boole

- Lecture 11: George Boole and nonbinary algebras

- Lecture 10: Boolean Algebra – from axioms to theorems

- Lecture 13: Shannon: from symbols to physics

- Lecture 12: Boolean is a useful language: functions and syntax boxes

- Lecture 14: Shannon: synthesis, simplicity, creativity

(4)

a

b

b c

a

b

c

Blue a = complement of a Red a = variable a

Quiz #10 – review

What is the Boolean function computed by the relay circuits

not connected

a

b c

b 1

2

(5)

Blue a = complement of a Red a = variable a

Quiz #10 – review

What is the Boolean function computed by the relay circuits

a

b c

b 1

(¯ a · b) + (b · b) + (b · ¯c)

= b + (¯ a · b) + (b · ¯c)

= b

= a + (b · c)

= ¯ a · (¯b + ¯c)

= ¯ a · ¯b + ¯a · ¯c

= ¯ a · ¯b · (c + ¯c) + ¯a · ¯c · (b + ¯b)

= ¯ a · ¯b · ¯c + ¯a · ¯b · c + ¯a · b · ¯c a + (b · c) + (a · b)

(a · b) + ¯c

= (a · b) · c

= (¯ a + ¯b) · c

= c · (¯a + ¯b)

= (c · ¯a) + (c · ¯b)

a · (a + (b · c · d · e · f)) = a

a · (a + (b · c)) = a

m = 4

(4 + 1)

²

4

²

= 2 · 4 + 1

(m + 1)

²

m

²

= 2 · m + 1 (¯ a · b) + (b · b) + (b · ¯c)

= b + (¯ a · b) + (b · ¯c)

= b

= a + (b · c)

= ¯ a · (¯b + ¯c)

= ¯ a · ¯b + ¯a · ¯c

= ¯ a · ¯b · (c + ¯c) + ¯a · ¯c · (b + ¯b)

= ¯ a · ¯b · ¯c + ¯a · ¯b · c + ¯a · b · ¯c a + (b · c) + (a · b)

(a · b) + ¯c

= (a · b) · c

= (¯ a + ¯b) · c

= c · (¯a + ¯b)

= (c · ¯a) + (c · ¯b)

a · (a + (b · c · d · e · f)) = a

a · (a + (b · c)) = a

m = 4

(4 + 1)

²

4

²

= 2 · 4 + 1

(m + 1)

²

m

²

= 2 · m + 1 (¯ a · b) + (b · b) + (b · ¯c)

= b + (¯ a · b) + (b · ¯c)

= b

= a + (b · c)

= ¯ a · (¯b + ¯c)

= ¯ a · ¯b + ¯a · ¯c

= ¯ a · ¯b · (c + ¯c) + ¯a · ¯c · (b + ¯b)

= ¯ a · ¯b · ¯c + ¯a · ¯b · c + ¯a · b · ¯c a + (b · c) + (a · b)

(a · b) + ¯c

= (a · b) · c

= (¯ a + ¯b) · c

= c · (¯a + ¯b)

= (c · ¯a) + (c · ¯b)

a · (a + (b · c · d · e · f)) = a

a · (a + (b · c)) = a

m = 4

(4 + 1)

²

4

²

= 2 · 4 + 1

(m + 1)

²

m

²

= 2 · m + 1

(6)

a

b

b c

a

b

c

Quiz #10 – review

What is the Boolean function computed by the relay circuits

not connected

2

(7)

Life and Stories

|s|

Shannon 1916-2001

(8)

Shannon 1916-2001

U of Michigan, undergrad, EE and Math (16): 1932-36 MIT MSc and PHD (20) : 1936-1940

Institute for Advanced Studies (24) : 1940-41 Bell Labs (25) : 1941-1958

MIT Professor (42) : 1958-1978

Married Elizabeth (Betty) Moore Shannon in 1949 (33)

Betty Shannon 1922-2017

(9)

Married Elizabeth (Betty) Moore Shannon in 1949

Children: Andrew and Margarita

Betty and Claude (64) in 1980

Claude (69) (and Betty?) with Caltech’s professor Yaser Abu-Mostafa at the 1985 Information Theory Symposium, Brighton, England

(10)

Shannon 1916-2001

logic design

information theory cryptography

computer chess and AI

computational juggling...

(11)

Shannon 1916-2001

logic design

information theory cryptography

computer chess and AI

computational juggling...

Posted on the class web site

(12)

A book on Shannon’s life A book with Shannon’s papers

(13)

MIT flight instructor to MIT president:

Shannon’s flying lessons at MIT, 1939 (23)

“I am convinced that Shannon is not only unusual but is in fact a near-genius of most unusual

promise.”

With the president’s permission he would ban Shannon from the cockpit: “such a life was not worth risking in a crash…”

”Somehow I doubt the advisability of urging a young man to refrain from flying or arbitrarily to take the opportunity away from him, on the ground of his being intellectually superior. I doubt whether it would be good for the development of his own character and

personality.”

MIT president (Karl Compton):

(14)

Shannon 1916-2001

Posted on the class web site

(15)

The Modern Relay?

A co-inventor of the modern relay - the transistor 1947-8 at Bell Labs... Together with Bardeen

and Brattain. 1956 Nobel Prize in Physics

William Shockley 1910 –1989

Caltech BS: 1928-1932 MIT PhD: 1932-1936

Blacker Hovse Blacker Hovse came into existence when its initial members met in the Athenaeum for its first meeting on Thursday, May 14, 1931

Blacker is 88 years old!

Source: Blacker house web page

Shockley was a junior in Physics

(16)

Connection Between

Boolean Calculus and Physical Circuits Shannon 1938

Boolean functions relay circuits relay circuits Boolean functions

Analysis (last time):

Synthesis (today):

(17)

Relay Circuits

synthesis

|s|

(18)

Boolean Calculus and Physical Circuits

Q: Boolean calculus to relay circuits?

How many relays with series-parallel?

(19)

a b

d e

a c e

b c d

Q: Boolean calculus to relay circuits?

Boolean Calculus and Physical Circuits

(20)

a b

d e

a c e

b c d

10 Q: Boolean calculus to relay circuits?

Boolean Calculus and Physical Circuits

(21)

Boolean Calculus and Physical Circuits

Less than 10 relays with series-parallel?

a b

d e

a c e

b c d

(22)

Boolean Calculus and Physical Circuits

Less than 10 relays with series-parallel?

a b

c

d e

b c d

e

(23)

a b c

d e

b c

e How many relays? 8

Can you compute

f

with less than 8 relays?

it using the algebra

5 relays?

Boolean Calculus and Physical Circuits

(30)

a b

c d e

How to efficiently construct “good”

relay circuits is an open problem

Boolean Calculus and Physical Circuits

(31)

Synthesis

trees

|s|

(32)

XOR of two Variables with Relays?

a

b b a

How many relays?

(33)

XOR of More Variables

How many relays? 12

b c

a

b c

a

b c

a

b

a c

Can we do better with series-parallel?

(34)

XOR of More Variables

How many relays? 10 b

c c b

a b a

(35)

b

c c b

a b a

Does it look familiar?

a=0

a=1

Idea?

odd parity

even parity

(36)

b

c c b

a b a

Does it look familiar?

a=0

a=1

The proof for magic boxes

using sub-functions

(37)

b c o

0 0

1 0

1 1

0 1 1

3-input binary s-box

can be divided to two 2-input s-boxes

0

0 0

1 0

1 1

1 0 0 1 a

0 0 0 0 1 1 1 1 a= 0 then

a= 1 then

odd parity

even parity

Shannon 1938

(38)

b c o 0 0

1 0

1 0 1 1

0 1 1 0 0 0

1 0

1 0 1 1

1 0 0 1 a

0 0 0 0 1 1 1 1 b

o

c b c

a a= 0 then a= 1 then

Shannon 1938

(39)

Representing

syntax boxes with

(40)

A syntax table / Boolean function as a binary decision tree

0 =1 =

? ?

A path corresponds to an entry in the syntax table

(0,0) (0,1) (1,0) (1,1)

(41)

? ?

0 =1 =

(0,0) (0,1) (1,0) (1,1) ab ^XOR(a,b)

00 0110 11

0 11 0

A syntax table / Boolean function

as a binary decision tree

(42)

0 =1 =

? ?

ab ^XOR(a,b)

00 0110 11

0 11

0 0 1 1 0

(0,0) (0,1) (1,0) (1,1)

A syntax table / Boolean function

as a binary decision tree

(43)

0 =1 =

? ?

ab ^{magic box}

00 0110 11

1 11

0 1 1 1 0

A syntax table / Boolean function

as a binary decision tree

(44)

0 1

(47)

b

c c b

a b a

Trees and Relay Circuits

0 1 1 0 1 0 0 1

(48)

0 1 1 0 1 0 0 1

b

c c b

a b a

How many relays? 10 Can we do better?

Trees and Relay Circuits

(49)

0 1 1 0 1 0 0 1

Can we do better?

Trees and Relay Circuits

not series-parallel 8 relays

(50)

0 = 1 =

1 1 0 0 1 0 1 1

Simplification of Boolean Functions sub-trees, karnaugh maps…

Key idea: No need to ‘climb a tree’ if we know the value earlier

simpler tree?

(51)

the order of variables is important

0

0 1 1

(54)

0 = 1 =

0 1

0 1 0 1 0 1 0 1

Simplification of Circuits,

sub-trees, karnaugh maps…

0

0 1 1

Observation:

the order of variables is important

(55)

binary decision trees + simplification

relay circuits

No need in Boolean algebra...

syntax boxes

(56)

Do simplifications of

binary decision trees

always provide the best

series-parallel solutions?

(57)

0 =1 =

1 0

1 1

ab ^{magic box}

00 0110 11

1 11 0

Find the simplest

series-parallel circuit

for the following syntax box

(58)

series-parallel solutions?

(63)

0 =1 =

? ab ^{magic box}

00 0110 11

1 11

0 1

Do binary decision trees always provide the best series-parallel solutions?

1

(64)

0 =1 = ab ^{magic box}

00 0110 11

1 11 0

Do binary decision trees always provide the best

series-parallel solutions? NO

?1 1

(65)

Synthesis

symmetric functions

|s|

(66)

Two questions about XOR

Why care about XOR?

Why is XOR ‘easy’ to implement?

(67)

The Boolean

Functions of the Adder

carry

2 symbol adder c

s

d1 d2

c

sum

(68)

2 symbol adder

c s

d1 d2 c

2 symbol adder c

s

d1 d2

c

A new design:

2 symbol adder

c s

d1 c d2

2 symbol adder

c s

d1 d2 c

How will you

help your mom to correctly

use the adders?

sum

carry

(69)

2 symbol adder

c s

d1 d2 c

2 symbol adder c

s

d1 d2

c

A new design:

2 symbol adder

c s

d1 c d2

2 symbol adder

c s

d1 d2 c

How will you

help your mom to correctly

use the adders?

(70)

carry

2 symbol adder c

s

d1 d2

c

sum

MAJ and XOR are

symmetric

Boolean functions

(71)

Symmetric Functions

Permuting the inputs does SYM not change the output

Definition: A Boolean function f is symmetric if

for an arbitrary permutation

AND, OR, MAJ and XOR are symmetric Boolean functions

(72)

abc XOR(a,b,c) 000

001010 011

0 11

100 0 101110 111

10 01

sum

abc XOR(a,b,c) 000

001010 100

0 11

011 1 101110 111

00 01

number of 1s

0 1

2 3

(73)

abc MAJ(a,b,c) 000

001010 011

0 00

100 1 101110 111

01 11

carry

abc MAJ(a,b,c) 000

001010 100

0 00

011 0 101110 111

11 11

number of 1s

0 1

2 3

(74)

Symmetric Functions

Definition: A Boolean function f is symmetric if

for an arbitrary permutation

Theorem: A Boolean function f(X) is symmetric

if and only if it is a function of the number of 1’s in X, namely |X|

(75)

Theorem: A Boolean function f(X) is symmetric

if and only if it is a function of the number of 1’s in X, namely |X|

f symmetric f a function of |X|

f symmetric f a function of |X|

abc XOR(a,b,c) 000

001010 011

0 11

100 0 101110 111

10 01

abc XOR(a,b,c) 000

001010 100

0 11

011 1 101110 111

00 01

number of 1s

0 1

2 3

(76)

abc XOR(a,b,c) 000

001010 011

0 11

100 0 101110 111

10 01

abc XOR(a,b,c) 000

001010 100

0 11

011 1 101110 111

00 01

0 1

2 3

Proof: f symmetric f a function of |X|

Given:

Need to prove:

However, ,

Hence,

Q

(77)

abc XOR(a,b,c) 000

001010 011

0 11

100 0 101110 111

10 01

abc XOR(a,b,c) 000

001010 100

0 11

011 1 101110 111

00 01

0 1

2 3

Proof: f a function of |X| f symmetric Given:

Need to prove:

However,

Q

(78)

Q1: How many symmetric Boolean functions of n variables?

A1: Theorem: The number of symmetric functions of

n

variables is:

(79)

Proof: f symmetric f a function of |X|

can be 0 or 1

*

functions

Q

01 2

n

**

*

* Symmetric

function table

n

variables is:

(80)

Q1: How many symmetric Boolean functions of n variables?

Q2: How can we efficiently implement symmetric functions with relay circuits?

A2: In Shannon’s MSc thesis and now!

n

variables is:

(81)

Circuits and Symmetric Functions

abc XOR(a,b,c) 000

001010 100

0 11

011 1 101110 111

00 01

0 1

2 3

Need to count the number of 1s What is the idea?

(82)

0

0 1

0 1 2

0 1 2 3

Circuits and Symmetric Functions

abc XOR(a,b,c) 000

001010 100

0 11

011 1 101110 111

00 01

0 1

2 3

(83)

0 1 2 3

Circuits and Symmetric Functions

abc XOR(a,b,c) 000

001010 100

0 11

011 1 101110 111

00 01

0 1

2 3

XOR

(84)

red = go right

0 1 2 3

The key: Symmetric functions are defined by the number of 1s

blue = go down

binomial tree?

1 1

1 2 1

1 3 3 1

1 4 6 4 1

What are we choosing?

Variables that are red =1 What are the numbers?

Not a tree

Possible assignments

(85)

binomial tree

Possible assignments for 4 variables:

Coefficients of:

Sum of Coefficients:

Possible assignments for n variables:

(86)

0 1 2 3

blue = go down red = go right How many edges/relays for n variables?

Number of round nodes is an arithmetic sum: n(n+1)/2 Number of edges is twice the number of nodes: n(n+1)

3(3+1)/2 = 6

12 binomial tree

The key: Symmetric functions are defined by the number of 1s

(87)

0 1 2 3

binomial tree

Fig. 25 in Shannon’s Thesis

(88)

Synthesis

summary

binary trees and binomial trees

|s|

(89)

Arbitrary Boolean functions

relay circuits

decision tree

0 1 1 0 1 0 0 1

Number of relays can be very LARGE: 2

ⁿ

(90)

Symmetric Boolean functions

relay circuits

binomial tree

0 1 2 3

Number of relays

about n

²

(91)

Synthesis

1 =

XOR

(96)

0 1 2

Implementing a 2-bit Adder

3

1 1 0 0

0

0 1

0 1 2

0 1 2 3

XOR 0 =

1 =

XOR

(97)

0 1 2

Implementing a 2-bit Adder

3

1 1 0 0

0

0 1

0 1 2

1 3

XOR 0 =

1 =

These two nodes are equivalent XOR c=1 results in XOR=1

(98)

0 1 2

Implementing a 2-bit Adder

3

1 1 0 0

1 3

XOR 0 =

1 =

XOR

(99)

0 1 2

0 1 2 3

0 1 0 1

2 3

MAJ 0 =

There is no 1 = reason to

check c here:

It is 1 for both values

(103)

Implementing a 2-bit Adder

0 1 2 3

0 1 0 1

2 3

MAJ 0 = 1 =

(104)

Implementing a 2-bit Adder

0 1 2 3

0 1 0 1

MAJ 0 = 1 =

2

(105)

Implementing a 2-bit Adder

0 1 2 3

0 1 0 1

Now what?

(109)

Relay Circuits for

the Sum and the Carry Functions

1

MAJ XOR

sum:

carry:

2 symbol adder c

s d1 d2

c

How many relays for

a 2 symbol adder? 10

(110)

This is Shannon’s Design

Shannon 1916-2001

1

MAJ XOR

(111)

This is Shannon’s Design

Shannon 1916-2001

1

MAJ MAJ XOR

XOR

(112)

Shannon 1916-2001

The amazing last

page of Shannon’s Thesis

binary arithmetic to Boolean algebra

Boolean algebra to optimized relay circuits

Key references on algebra (in the context of logic)

(113)

Shannon 1916-2001

The amazing last

page of Shannon’s Thesis

binary arithmetic to Boolean algebra

Boolean algebra to optimized relay circuits

Key references on algebra (in the context of logic) Betty and “Claude” Shannon,

U of Michigan

(114)

Pre-Quiz

(115)

(116)

(117)

Quiz time

(118)

Quiz #11 – 10min

There are four possible Boolean functions of a single variable:

0 1

0 0 x 0

0 1

1 1 x 1

0 1

0 1 x x

0 1

?

? x

Construct a multiple terminal relay circuit that implements the four functions.

Please minimize the number of relays.

0 1

1 0 x x-