IST 4 Information and Logic

(1)

IST 4

Information and Logic

(2)

mon tue wed thr fri

3 _M₁ 1

10 _M₁

17 1 2 ^M²

24 2

1 ^M²

8 3

15 3 4

22 4 5

29

5 5

x= hw#x out x= hw#x due

Mx= MQx out Mx= MQx due

midterms

oh oh

oh

oh oh

oh

oh oh oh

oh

oh oh

oh = office hours ^oh

T

= today

T oh

oh

sun

CP1

CP2 CP

CP = challenge problem

(3)

Today’s lecture is based on Shannon’s MSc

thesis

|s|

Shannon 1916-2001

(4)

Shannon 1916-2001

Claude Elwood Shannon was born in Petoskey, Michigan, on April 30, 1916. The first sixteen

years of Shannon's life were spent in Gaylord, Michigan His father, Claude Sr. (1862-1934), was a businessman and for a period a Judge of Probate

His mother, Mabel Wolf Shannon (1880-1945), was a language teacher and for a number of years principal of Gaylord High School

George Boole (1815-1864):

generation of Shannon’s grandfather

54

36

(5)

Shannon 1916-2001

U of Michigan, undergrad, EE and Math (16): 1932-36 MIT MSc and PHD (20) : 1936-1940

Institute for Advanced Studies (24) : 1940-41 Bell Labs (25) : 1941-1958

MIT Professor (42) : 1958-1978

Married Elizabeth (Betty) Moore Shannon in 1949 (33)

(6)

Married Elizabeth (Betty) Moore Shannon in 1949

Children: Andrew and Margarita

Betty and Claude in 1980

Claude (and Betty?) with Caltech’s professor Yaser Abu-Mostafa at the 1985 Information Theory Symposium, Brighton, England

(7)

Shannon 1916-2001

logic design

information theory cryptography

computer chess and AI

computational juggling...

(8)

Shannon 1916-2001

Posted on the class web site

(9)

The Modern Relay?

A co-inventor of the modern relay - the transistor 1947-8 at Bell Labs... Together with Bardeen

and Brattain. 1956 Nobel Prize in Physics

William Shockley 1910 –1989

Caltech BS: 1928-1932 MIT PhD: 1932-1936

Blacker Hovse Blacker Hovse came into existence when its initial members met in the Athenaeum for its first meeting on Thursday, May 14, 1931

Blacker is 86 years old!

Source: Blacker house web page

Shockley was a junior in Physics

(10)

Connection Between

Boolean Calculus and Physical Circuits Shannon 1938

Boolean functions relay circuits relay circuits Boolean functions

Analysis (last time):

Synthesis (today):

(11)

Relay Circuits

synthesis

|s|

(12)

Boolean Calculus and Physical Circuits

Q: Boolean calculus to relay circuits?

How many relays with series-parallel?

(13)

a b

d e

a c e

b c d

Q: Boolean calculus to relay circuits?

(14)

a b

d e

a c e

b c d

10 Q: Boolean calculus to relay circuits?

(15)

Less than 10 relays with series-parallel?

a b

d e

a c e

b c d

(16)

Less than 10 relays with series-parallel?

a b

c

d e

b c d

e

(17)

a b c

d e

b c

e How many relays? 8

Can you compute f with less than 8 relays?

Lower bound on the number on the number of relays???

5 relays?

(18)

5 relays?

(19)

a b

c d e

5 relays?

(20)

a b

c d e

5 relays?

(21)

a b

c d e

5 relays?

(22)

a b

c d e

5 relays?

(23)

a b

c d e

This circuit is not series-parallel and we do not have algorithms for synthesis it using the algebra

5 relays?

(24)

a b

c d e

How to efficiently construct “good”

relay circuits is an open problem

(25)

Synthesis

trees

|s|

(26)

XOR of two Variables with Relays?

a

b b a

How many relays?

(27)

XOR of More Variables

How many relays? 12

b c

a

b c

a

b c

a

b

a c

Can we do better with series-parallel?

(28)

XOR of More Variables

How many relays? 10 b

c c b

a b a

(29)

b

c c b

a b a

Does it look familiar?

a=0

a=1

Idea?

odd parity

even parity

(30)

b

c c b

a b a

Does it look familiar?

a=0

a=1

The proof for magic boxes

using sub-functions

(31)

b c o

0 0

1 0

1 1

0 1 1

3-input binary s-box

can be divided to two 2-input s-boxes

0

0 0

1 0

1 1

1 0 0 1 a

0 0 0 0 1 1 1 1 a= 0 then

a= 1 then

odd parity

even parity

Shannon 1938

(32)

b c o 0 0

1 0

1 0 1 1

0 1 1 0 0 0

1 0

1 0 1 1

1 0 0 1 a

0 0 0 0 1 1 1 1 b

o

c b c

a a= 0 then a= 1 then

Shannon 1938

(33)

Representing

syntax boxes with

(34)

A syntax table / Boolean function as a binary decision tree

0 = 1 =

? ?

A path corresponds to an entry in the syntax table

(0,0) (0,1) (1,0) (1,1)

(35)

? ?

0 = 1 =

(0,0) (0,1) (1,0) (1,1) ab ^XOR(a,b)

00 01 10 11

0 1 1 0

(36)

0 = 1 =

? ?

? ? ab ^XOR(a,b)

00 01 10 11

0 1 1

0 0 1 1 0

(0,0) (0,1) (1,0) (1,1)

(37)

0 = 1 =

? ?

? ? ab magic box

00 01 10 11

1 1 1

0 1 1 1 0

(38)

0 1

0 = 1 =

1 0 1 0 0 1

Can the tree representation help with the synthesis of relay circuits?

0 1 1 0 1 0 0 1

(42)

0 1 1 0 1 0 0 1

b

c c b

a b a

How many relays? 10 Can we do better?

Trees and Relay Circuits

(43)

0 1 1 0 1 0 0 1

Can we do better?

Trees and Relay Circuits

not series-parallel 8 relays

(44)

0 = 1 =

1 1 0 0 1 0 1 1

Simplification of Boolean Functions sub-trees, karnaugh maps…

Key idea: No need to ‘climb a tree’ if we know the value earlier

simpler tree?

(45)

0 = 1 = 1 1

1 0 1

0 0 1 0 1 1

Simplification of Circuits, sub-trees, karnaugh maps…

Key idea: No need to ‘climb a tree’ if we

know the value earlier

(46)

0 = 1 =

0 1 0 1 0 1 0 1

Simplification of Circuits, sub-trees, karnaugh maps…

Observation:

the order of variables is important

(47)

0 = 1 =

0 1 0 1 0 1 0 1

Simplification of Circuits, sub-trees, karnaugh maps…

Observation:

the order of variables is important

0

0 1 1

(48)

0 = 1 =

0 1

0 1 0 1 0 1 0 1

Simplification of Circuits, sub-trees, karnaugh maps…

0

0 1 1

Observation:

the order of variables is important

(49)

binary decision trees + simplification

relay circuits

No need in Boolean algebra...

syntax boxes

(50)

Do binary decision trees

always provide the best

series-parallel solutions?

(51)

0 = 1 =

1 0 1 1

ab magic box 00

01 10 11

1 1 1 0

Find the simplest

Do binary decision trees always provide the best series-parallel solutions?

1

(58)

0 = ab magic box 1 =

00 01 10 11

1 1 1 0

Do binary decision trees always provide the best

series-parallel solutions? NO

? 1 1

(59)

Synthesis

symmetric functions

|s|

(60)

Two questions about XOR

Why care about XOR?

Why is XOR ‘easy’ to implement?

(61)

The Boolean

Functions of the Adder

carry

2 symbol adder c

s d1 d2

c

sum

(62)

2 symbol adder

c s

d1 d2 c 2 symbol adder

c

s d1 d2

c

A new design:

2 symbol adder

c s

d1 c d2 2 symbol adder

c s

d1 d2 c

How will you help your mom to correctly

use the adders?

sum

carry

(63)

2 symbol adder

c s

d1 d2 c 2 symbol adder

c

s d1 d2

c

A new design:

2 symbol adder

c s

d1 c d2 2 symbol adder

c s

d1 d2 c

How will you help your mom to correctly

use the adders?

(64)

carry

2 symbol adder c

s d1 d2

c

sum

MAJ and XOR are

symmetric

Boolean functions

(65)

Symmetric Functions

Permuting the inputs does SYM not change the output

Definition: A Boolean function f is symmetric if

for an arbitrary permutation

AND, OR, MAJ and XOR are symmetric Boolean functions

(66)

abc XOR(a,b,c) 000

001 010 011

0 1 1

100 0 101 110 111

1 0 0 1

sum

abc XOR(a,b,c) 000

001 010 100

0 1 1

011 1 101 110 111

0 0 0 1

number of 1s

0 1

2 3

(67)

abc MAJ(a,b,c) 000

001 010 011

0 0 0

100 1 101 110 111

0 1 1 1

carry

abc MAJ(a,b,c) 000

001 010 100

0 0 0

011 0 101 110 111

1 1 1 1

number of 1s

0 1

2 3

(68)

Symmetric Functions

Definition: A Boolean function f is symmetric if

for an arbitrary permutation

Theorem: A Boolean function f(X) is symmetric if and only if it is a function of the number of 1’s in X, namely |X|

(69)

Theorem: A Boolean function f(X) is symmetric if and only if it is a function of the number of 1’s in X, namely |X|

f symmetric f a function of |X|

f symmetric f a function of |X|

abc XOR(a,b,c) 000

001 010 011

0 1 1

100 0 101 110 111

1 0 0 1

abc XOR(a,b,c) 000

001 010 100

0 1 1

011 1 101 110 111

0 0 0 1

number of 1s

0 1

2 3

(70)

abc XOR(a,b,c) 000

001 010 011

0 1 1

100 0 101 110 111

1 0 0 1

abc XOR(a,b,c) 000

001 010 100

0 1 1

011 1 101 110 111

0 0 0 1

0 1

2 3

Proof: f symmetric f a function of |X|

Given:

Need to prove:

However, ,

Hence,

Q

(71)

abc XOR(a,b,c) 000

001 010 011

0 1 1

100 0 101 110 111

1 0 0 1

abc XOR(a,b,c) 000

001 010 100

0 1 1

011 1 101 110 111

0 0 0 1

0 1

2 3

Proof: f a function of |X| f symmetric Given:

Need to prove:

However,

Q

(72)

Q1: How many symmetric Boolean functions of n variables?

A1:

Theorem: The number of symmetric functions of

n

variables is:

(73)

Proof: f symmetric f a function of |X|

can be 0 or 1

*

functions

Q

0 1 2

n

* *

*

* Symmetric

function table

A1:

n

variables is:

(74)

Q1: How many symmetric Boolean functions of n variables?

Q2: How can we efficiently implement symmetric functions with relay circuits?

A2:

In Shannon’s MSc thesis and now!

A1:

n

variables is:

(75)

Circuits and Symmetric Functions

abc XOR(a,b,c) 000

001 010 100

0 1 1

011 1 101 110 111

0 0 0 1

0 1

2 3

Need to count the number of 1s What is the idea?

(76)

0

0 1

0 1 2

0 1 2 3

Circuits and Symmetric Functions

abc XOR(a,b,c) 000

001 010 100

0 1 1

011 1 101 110 111

0 0 0 1

0 1

2 3

(77)

0 1 2 3

Circuits and Symmetric Functions

abc XOR(a,b,c) 000

001 010 100

0 1 1

011 1 101 110 111

0 0 0 1

0 1

2 3

XOR

(78)

red = go right

0 1 2 3

The key: Symmetric functions are defined by the number of 1s

blue = go down

binomial tree?

1 1

1 2 1

1 3 3 1

1 4 6 4 1

What are we choosing?

Variables that are red =1 What are the numbers?

Not a tree

(79)

0 1 2 3

blue = go down red = go right How many edges/relays for n variables?

Number of round nodes is an arithmetic sum: n(n+1)/2 Number of edges is twice the number of nodes: n(n+1)

3(3+1)/2 = 6

12 binomial tree

The key: Symmetric functions are defined by the number of 1s

(80)

Synthesis

summary

binary trees and binomial trees

|s|

(81)

Arbitrary Boolean functions

relay circuits

decision tree

0 1 1 0 1 0 0 1

Number of relays can be very LARGE: 2

ⁿ

(82)

Symmetric Boolean functions

relay circuits

binomial tree

0 1 2 3

Number of relays

about n

²

(83)

Synthesis

1 =

XOR

(88)

0 1 2

Implementing a 2-bit Adder

3

1 1 0 0

0

0 1

0 1 2

0 1 2 3

XOR 0 =

1 =

XOR

(89)

0 1 2

Implementing a 2-bit Adder

3

1 1 0 0

0

0 1

0 1 2

1 3

XOR 0 =

1 =

These two nodes are equivalent XOR c=1 results in XOR=1

(90)

0 1 2

Implementing a 2-bit Adder

3

1 1 0 0

1 3

XOR 0 =

1 =

XOR

(91)

0 1 2

0 1 2 3

0 1 0 1

2 3

MAJ 0 = 1 =

There is no reason to

check c here:

It is 1 for both values

(95)

Implementing a 2-bit Adder

0 1 2 3

0 1 0 1

2 3

MAJ 0 = 1 =

(96)

Implementing a 2-bit Adder

0 1 2 3

0 1 0 1

MAJ 0 = 1 =

2

(97)

Implementing a 2-bit Adder

0 1 2 3

0 1 0 1

Now what?

(101)

Relay Circuits for

the Sum and the Carry Functions

1 XOR MAJ sum:

carry:

2 symbol adder c

s d1 d2

c

How many relays for

a 2 symbol adder? 10

(102)

This is Shannon’s Design

Shannon 1916-2001

1 XOR MAJ

(103)

This is Shannon’s Design

Shannon 1916-2001

1 XOR MAJ MAJ

XOR

(104)

Shannon 1916-2001

The amazing last

page of Shannon’s Thesis

binary arithmetic to Boolean algebra

Boolean algebra to optimized relay circuits

Key references on algebra (in the context of logic)

(105)

Quiz time

(106)

a b c

d d

1.  What is the Boolean function that is computed by the following relay circuit? Boolean formula 2. Show the simplest relay circuit (smallest

number of relays) that can compute this function. Relay circuit

Why is it the simplest circuit? Explanation

Quiz #8 – 10min

Show your work!