IST 4
Information and Logic
MQ1 grades were emailed
please let the TAs know if you submitted MQ1 and did not get the email
HW1 will be returned today average grade is A
Everyone has a gift! MQ2
Due Tuesday 5/2/2017 by 10pm
Please email PDF
lastname-firstname.pdf
HW#2 due Thursday, 4/27/2017
mon tue wed thr fri
3 M1 1
10 M1
17 1 2 M2
24 2
1 M2
8 3
15 3 4
22 4 5
29
5 5
x= hw#x out x= hw#x due
Mx= MQx out Mx= MQx due
midterms
oh oh
oh oh
oh oh
oh
oh
oh oh
oh
oh oh oh
oh
oh oh
oh = office hours oh
T
= today
T oh
oh
oh
oh
sun
The geography of innovation We need the whole world!
Source: Wikipedia Africa
Babylonia and Egypt Greece
Maya
Central Asia China and India
Europe USA
Source: Wikipedia Africa
Babylonia and Egypt Greece
Maya
Central Asia China and India
Europe USA
Fibonacci 1170-1250
Algorizmi 780-850 Leibniz
1646-1716
Euclid, 300BC
Last time...
Euclid,300BC
GCD = 5
Last time...
Fibonacci 1170-1250 Algorizmi
780-850
Leibniz 1646-1716
base 10 base 10
in Europe The binary
Algorithms – Syntax Manipulation
2 symbol adder
digit 1 digit 2
carry
sum
carry
Algorithms – Syntax Manipulation
2 symbol adder
8 6
digit 1 digit 2
carry
sum
0 carry
Algorithms – Syntax Manipulation
2 symbol adder
8 6
4
digit 1 digit 2
carry
sum
1 0 carry
Arithmetic Boxes binary
Leibniz 1646-1716
The binary
2 symbol adder
digit 1 digit 2
carry
sum
carry
Adding Bits
3 bits to 2 bits
Count the number
of 1s and represent in binary
Adding Bits
2 symbol adder
0 0
0
digit 1 digit 2
carry
sum
0 0 carry
3 bits to 2 bits
Count the number
of 1s and represent in binary
2 symbol adder
1 0
1
digit 1 digit 2
carry
sum
0 0 carry
Adding Bits
3 bits to 2 bits
Count the number
of 1s and represent in binary
2 symbol adder
1 1
0
digit 1 digit 2
carry
sum
1 0 carry
Adding Bits
3 bits to 2 bits
Count the number
of 1s and represent in binary
2 symbol adder
1 0
0
digit 1 digit 2
carry
sum
1 1 carry
Adding Bits
3 bits to 2 bits
Count the number
of 1s and represent in binary
2 symbol adder
0 1
0
digit 1 digit 2
carry
sum
1 1 carry
Adding Bits
3 bits to 2 bits
Count the number
of 1s and represent in binary
2 symbol adder
1 1
1
digit 1 digit 2
carry
sum
1 1 carry
Adding Bits
3 bits to 2 bits
Count the number
of 1s and represent in binary
2 symbol adder
1 1
1
digit 1 digit 2
carry
sum
1 1 carry
when / where was
‘the bit’ invented?
3 bits to 2 bits
Count the number
of 1s and represent in binary
Shannon 1916-2001
when / where was
‘the bit’ invented?
The choice of a logarithmic base corresponds to the choice of a unit for measuring information. If the base 2 is used the
resulting units may be called binary digits, or more brie3ly bits, a word suggested by J. W. Tukey.
Shannon 1916-2001 Tukey
1915-2000
Algorithms – Syntax Manipulation
2 symbol adder c
s d1 d2
c
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
sum
carry
0 1 0 0 1 1 1 2
0 0 1
0 1 2 1 2 3
1
Algorithms – Syntax Manipulation
2 symbol adder 1
0
1 1
0
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
Algorithms – Syntax Manipulation
2 symbol adder 1
0
1 0
1
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
2 symbol adder c
c
1 1
c c 2 symbol adder c
1 0
c c 2 symbol adder c
0 0
c c 2 symbol adder c
1 1
0
1101 + 1001 = ??
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
2 symbol adder c
c
1 1
c c 2 symbol adder c
1 0
c c 2 symbol adder c
0 0
c 1 2 symbol adder 0
1 1
0
1101 + 1001 = ??
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
2 symbol adder c
c
1 1
c c 2 symbol adder c
1 0
c 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
1101 + 1001 = ??
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
2 symbol adder c
c
1 1
c 0 2 symbol adder 1
1 0
0 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
1101 + 1001 = ??
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
2 symbol adder 1
0
1 1
0 0 2 symbol adder 1
1 0
0 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
1101 + 1001 = ??
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
2 symbol adder 1
0
1 1
0 0 2 symbol adder 1
1 0
0 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
1101 + 1001 = 10110 13 + 9 = 22
2 symbol adder 1
0
1 1
0 0 2 symbol adder 1
1 0
0 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
1101 + 1001 = 10110 13 + 9 = 22
8 bit adder
1 1 0 1 1 0 0 1
1 0 1 1 0
a deeper idea?
1 1 0 1 1 0 0 1
1 0 1 1 0
8 input bits
5 output bits
28 input combinations
Can be implemented with a table with 5 x 256 bits ???bits???
what are the trade-offs?
2 symbol adder 1
0
1 1
0 0 2 symbol adder 1
1 0
0 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
0 1 0 0 1 1 1 0
0
0 1 0 1 0 1 0 1
1
0 1 0 0 0 1 0 1
0
0 1 0 0 1 1 1 1
1
sum
carry what are the trade-offs?
2 symbol adder 1
0
1 1
0 0 2 symbol adder 1
1 0
0 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
1 0 0
1 1 1 0
1
a table with 8 bits
8 tables with 8 bits
a table with 8 bits
2 symbol adder 1
0
1 1
0 0 2 symbol adder 1
1 0
0 0 2 symbol adder 1
0 0
1 1 2 symbol adder 0
1 1
0
a deeper idea?
Memory-only requires 5 x 256 bits = 20 x 64 bits Memory and logic require 8 8-bit tables = 64 bits
Idea: Logic (algorithms) is
a language that helps in optimizing memory Idea: Understanding is
is a method for remembering with a small memory…
and much more...
a deeper idea?
Which language is more mature?
Physics Biology
We need to remember
• The building blocks
• The algorithms
Algorithm
A procedure to build
‘everything’ from a set of building blocks
There is a trade-off between
the size of building blocks: memories
and the complexity of the algorithm:
composition of the memories
Algorithm
A procedure to build
‘everything’ from a set of building blocks
Syntax Boxes BIG
from smallRemembering a large memory
with a composition of small memories
S-Box
a b
o inputs
outputs
a b o
Binary Syntax Boxes (s-box)
0 0
0 inputs
outputs
a b o
0 0 0
Binary Syntax Boxes (s-box)
0 1
1 inputs
outputs
a b o
0 0
1 0
0 1
Binary Syntax Boxes (s-box)
1 0
1 inputs
outputs
a b o
0 0
1 0
1 0
0 1 1
Binary Syntax Boxes (s-box)
1 1
1 inputs
outputs
a b o
0 0
1 0
1 0
1 1
0 1 1 1 Suggest a name for this memory box?
a b
o
a b o
0 0
1 0
1 0
1 1
0 1 1 1
o = max (a,b)
o = max (a,b) Suggest a name for this memory box?
Can we remember
max (a,b,c) with max(a,b)?
a b
c
Composition:
build big s-boxes
from small s-boxes o = max (a,b)
o = max (a,b,c) Can we remember
max (a,b,c) with max(a,b)?
a b
o
o = max (a,b) Can we remember w
with the max s-box?
a b w
0 0
1 0
1 0
1 1
0 1 1 0
?
NO Why not?
a b o
0 0
1 0
1 0
1 1
0 1 1 1 o = max (a,b)
How to compute the following (XOR)?
Big
s-boxa b
w
a b
0 0
1 0
1 0
1 1
0 1 1 0
?
Composition:
build big s-boxes
from small s-boxes The output of every small s-box is bigger or equal to its inputs
The output of the big
s-box must be bigger or equal to its inputs NO Why not?
w
Can we remember w with the max s-box?
A Magic Box?
universality
A Magic (Universal) Box
a b
a b m
0 0
1 0
1 0
1 1
1 1 1 0
A binary s-box that can
compute any binary s-box?
M
m
Can the m-box compute any
(single input) binary syntax box?
How many different binary 1-input s-boxes? 22 = 4
y 0 1
o 0 0 y
0 1
o 1 1
y 0 1
o 0
1 y
0 1
o 1 0
y
1-y 0
1
a b m
0 0
1 0
1 0
1 1
1 1 1 0
a b m
0 0
1 0
1 0
1 1
1 1 1 0
1-y
M o y
a b m
0 0
1 0
1 0
1 1
1 1 1 0
M o y
1
1-y
a b m
0 0
1 0
1 0
1 1
1 1 1 0
x y o
0 0
1 0
1 0
1 1
1 1 1 1 M
o y
1
1-y y
M
Can the m-box compute any
(single input) binary syntax box?
How many different binary 1-input s-boxes? 22 = 4
y 0 1
o 0 0 y
0 1
o 1 1
y 0 1
o 0
1 y
0 1
o 1 0
Y
1-Y 0
1
a b m
0 0
1 0
1 0
1 1
1 1 1 0
Syntax Boxes
universality for two inputs
a b
a b m
0 0
1 0
1 0
1 1
1 1 1 0
M
m
Can the m-box compute any
(two input) binary syntax box?
x y
0 0
1 0
1 0
1 1
o
*
*
*
* How many different binary 2-input
s-boxes?
How will you prove it?
24 = 16
min(x,y)
1-y 1
max(x,y)
4 Useful Boxes
A Magic Box
a b
a b m
0 0
1 0
1 0
1 1
1 1 1 0
Can you compute the
following with the magic box?
M
m
x y
0 0
1 0
1 0
1 1
o 0 0 0 1 min(x,y)
A Magic Box
a b
a b m
0 0
1 0
1 0
1 1
1 1 1 0
Can you compute the
following with the magic box?
M
m
x y
0 0
1 0
1 0
1 1
o 0 0 0 1
Hint: min(x,y)
A Magic Box
x y
a b m
0 0
1 0
1 0
1 1
1 1 1 0
M
x y
0 0
1 0
1 0
1 1
o 0 0 0 M 1
o 1
0
min(x,y)
A Magic Box
x y
a b m
0 0
1 0
1 0
1 1
1 1 1 0
M
x y
0 0
1 0
1 0
1 1
o 0 0 0 M 1
o 0
1
min(x,y)
a b
a b m
0 0
1 0
1 0
1 1
1 1 1 0
Can you compute the
following with the m-box?
M
m
x y
0 0
1 0
1 0
1 1
o 0
1 1 1
Hint: max(x,y)
x
a b m
0 0
1 0
1 0
1 1
1 1 1 0
x y
0 0
1 0
1 0
1 1
o 0
1 1 M 1
o
y
M M
max(x,y)
x
a b m
0 0
1 0
1 0
1 1
1 1 1 0
x y
0 0
1 0
1 0
1 1
o 0
1 1 M 1
o
y
M M
0
1 1
0
0
max(x,y)
x
a b m
0 0
1 0
1 0
1 1
1 1 1 0
x y
0 0
1 0
1 0
1 1
o 0
1 1 M 1
o
y
M M
0
0 1
1
1
max(x,y)
x
a b m
0 0
1 0
1 0
1 1
1 1 1 0
x y
0 0
1 0
1 0
1 1
o 0
1 1 M 1
o
y
M M
1
0 0
1
1
max(x,y)
min(x,y)
1-y 1
max(x,y)
Need to prove:
any (two input) binary syntax box can be computed by
the 4 Useful Boxes
x y o
0 0
1 0
1 0
1 1
*
*
*
An Arbitrary Two Input Box
* Two 1-input boxes!
x= 0 then x= 1 then
What are the possible values of 0
0
0 1
1 0
1 1
x y o
0 0
1 0
1 0
1 1
*
*
*
A Selector Box
* x= 0 then
x= 1 then
selector
o
x
x y o
0 0
1 0
1 0
1 1
*
*
*
A Selector Box
* x= 0 then
x= 1 then
selector
o
0
x y o
0 0
1 0
1 0
1 1
*
*
*
A Selector Box
* x= 0 then
x= 1 then
selector
o
1
x y o
0 0
1 0
1 0
1 1
*
*
*
A Selector Box
* x= 0 then
x= 1 then
selector
o
x How can you compute this box?
max(a,b)
min(a,b) min(a,b)
o x= 0 then
x= 1 then
1-x x
A Selector Box
max(a,b)
min(a,b) min(a,b)
o x= 0 then
x= 1 then
0 1
0
A Selector Box
max(a,b)
min(a,b) min(a,b)
o x= 0 then
x= 1 then
1 0
0
QED A Selector Box
x y o
0 0
1 0
1 0
1 1
0 1 1
How to compute the following (XOR)?
0
selector
o
x
?
? x= 0 then
x= 1 then
x y o
0 0
1 0
1 0
1 1
0 1 1 0
selector
o
x 1-y
y
How to compute the following (XOR)?
x= 0 then x= 1 then
x y o
0 0
1 0
1 0
1 1
0 1 1 0
selector
o
x 1-y
y x= 0 then
x= 1 then
y
M
Given a 2-input binary box that can compute any 2-input binary box
Can it compute any 3-input binary box?
Does the magic continue?
a b
M
o
x z
o y
x y o
0 0
1 0
1 0
1 1
*
*
*
3-input binary s-box
*
0 0
1 0
1 0
1 1
*
*
*
* z
0 0 0 0 1 1 1 1 How many different
binary 3-input s-boxes?
28 = 256
x y o
0 0
1 0
1 0
1 1
*
*
*
3-input binary s-box
*
0 0
1 0
1 0
1 1
*
*
*
* z
0 0 0 0 1 1 1 1 Two 2-input boxes!
z= 0 then z= 1 then
x y o 0 0
1 0
1 0 1 1
*
*
*
* 0 0
1 0
1 0 1 1
*
*
*
* z
0 0 0 0 1 1 1 1 x
selector
o
y x y
z
z= 0 then z= 1 then All 2 input boxes can be computed with M
Given a magical box for any 2-input binary box
We proved that it is magical for any 3-input binary box!
a b
M
o o
....
Is it magical for any n-input binary box????
YES!!!!
Proof by induction on the number of inputs
Does the magic continue?
x
selector
o
(n-1) inputs (n-1) inputs
y x y
z
z= 0 then z= 1 then
Assume all boxes with (n-1) inputs can be computed with M
....
....
Given a set of building blocks:
What can/cannot be constructed?
Efficiency and complexity Feasibility
Time: How long will it take to complete the construction?
Size: If feasible, how many blocks are needed?
Questions about
algorithms
and building blocks?
Magic boxes do not know arithmetic…
Leibniz: A universal language
Instead of “thinking” let’s “calculate with magic boxes”
....
Gottfried Leibniz 1646-1716
Magic boxes do not know arithmetic…
Instead of “thinking” let’s “calculate with magic boxes”
smile recognition 0
....
1
Instead of “thinking” let’s “calculate with magic boxes”
Can computers outperform the human brain?
Or is it just a big memory monkey implemented with small monkeys ?
Gottfried Leibniz 1646-1716
Leibniz was the first person to understand that computation
is not just arithmetic
Universal Language for Reasoning The founder of
Information Science
Quiz time
x
a b m
0 0
1 0
1 0
1 1
1 1 1 0
x y
0 0
1 0
1 0
1 1
o
?
?
?
? M
y
M
M Quiz #4 – 10min
Specify the output of the circuit
o
o
