IST 4 Information and Logic

(1)

IST 4

Information and Logic

Claude Shannon April 30, 1916 - 2001

(2)

mon tue wed thr fri

28 _M₁ 1

4 _M₁

11 1 2 ^M²

18 2

25 _M₂

2 3

9 3 4

16 4 5

23

30 5

x= hw#x out x= hw#x due

Mx= MQx out Mx= MQx due

midterms

oh oh

oh

oh oh

oh

oh oh oh

oh

oh oh

oh = office hours ^oh

T

= today

T oh

(3)

Boolean Proofs: Axioms + Fun

(4)

A1-A4

L1 T1

T2 T3

L2

T4 T0

Proof

Dependency

(5)

Syllogism to Algebra George Boole, 1847

George Boole 1815 –1864

(6)

George Boole Early Days

George Boole 1815-1864

Born in Lincoln, England an industrial town

His father was a shoemaker with a passion for mathematics and science

When George was 8 he surpassed his father’s

knowledge in mathematics By age 14 he was fluent in Latin, German, French, Italian and English… and algebra…

When his was 15 he had to go to work to support his family, he became a math teacher in the Wesleyan Methodist academy in Doncaster (40 miles…)

Lost his job after two years….

Lost two more teaching jobs…

When he was 20 he opened his own school in his hometown - Lincoln

(7)

source: wikipedia

In 1841 (26) he published three papers in the newly established Cambridge Mathematical Journal (edited by DG).

In 1844 (29) he published “On a General Method of Analysis”; he considered it to be his best paper. This paper won the first (newly established) Gold medal for Mathematics awarded by Royal Society

In 1846 (31) he applied for a professor position in the newly established Queen’s College – 3 campuses in Ireland

In 1847 (32) ‘while waiting to hear from Ireland’, he published “The Mathematical Analysis of Logic”

George Boole Early Career

(8)

In 1847 (32) ‘while waiting to hear from Ireland’, he published “The Mathematical Analysis of Logic”

In 1849 (34), his was offered a position of the first professor of mathematics at Queen’s college at Cork

He married Mary Everest (1832- 1916) in 1855 (23,40) and they had five daughters

Niece of George Everest (Mt. Everest…) led the expedition to map the Himalayas

8848 m 29,029 ft

George Boole Ireland

source: wikipedia

In 1864, died of pneumonia (49)

(9)

1849-1864, taught at

at Queen’s college in Cork

George Boole Geography

source: wikipedia

(10)

Cork, Ireland Source: www.flickr.com

1849-1855:

lived here until got married

(11)

Boole was never a student at a university…

only a professor…

Leibniz never held an academic position…

George Boole 1815-1864 Gottfried Leibniz

1646-1716

(12)

It is all about

Algorizmi

780-850AD Gottfried Leibniz

1646-1716

???

Binary

Boolean

(13)

Boolean Algebra

Boolean is

^not

Binary...

(14)

Two-valued Boolean Algebra

Boolean Algebra: set of elements B={0,1},

two binary operations OR and AND xy ^OR(x,y)

00 01 10 11

0 1 1 1

xy ^AND(x,y)

00 01 10 11

0 0 0 1

0 iff both x and y are 0 1 iff both x and y are 1 We proved it is a Boolean algebra

(15)

Four-valued Boolean Algebra

Two-valued Boolean Algebra:

set of elements B={0,1},

two binary operations OR and AND Four-valued Boolean Algebra:

set of elements

??

two binary operations

??

^and

??

(16)

Elements:

0-1 vectors:

(10)

(00) (01) (11) operations?

(17)

+

Elements are: (00), (11), (10), (01)

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

(18)

+

Elements are: (00), (11), (10), (01)

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

Bitwise OR

(19)

+

Elements are: (00), (11), (10), (01)

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

Bitwise OR

⁰⁰₀₁^xy ^OR(x,y)

10 11

0 1 1 1

(20)

Elements are: (00), (11), (10), (01)

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

(21)

Elements are: (00), (11), (10), (01)

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

Bitwise AND

(22)

Elements are: (00), (11), (10), (01)

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

Bitwise AND

⁰⁰₀₁^xy ^AND(x,y)

10 11

0 0 0 1

(23)

Elements:

Operations:

(10)

(00) (01) (11)

Bitwise AND Bitwise OR

Is it a Boolean algebra?

0 1

Bitwise Complement

(24)

+

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

(25)

+

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

(26)

+

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

(27)

+

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

We can prove it!

(28)

Elements:

Operations:

(10)

(00) (01) (11)

It is a Boolean algebra!

0 1

In general: 0-1 vectors a Boolean algebra?

(29)

Elements:

Operations:

(001)

(000) (010) (111)

0 1

True for a two-valued Boolean algebra True for any Boolean algebra with

0-1 vectors and bitwise OR and AND??

0-1 vectors Boolean algebra

(011) (100) (101) (110)

n=3

n=1

arbitrary finite n

(30)

|s|

Idea 1:

Any identity that is true for 0-1 is true for 0-1 vectors with bitwise OR/AND

Idea 2:

Axioms are identities

True for 0-1 à true for 0-1 vectors

(31)

The 0-1 Theorem

0-1 Theorem:

An identity is true for 0-1 vectors with bitwise OR and AND if and only if

it is true for two valued (0-1) with bitwise OR and AND

Proof: The easy direction

Assume an identity true for 0-1 vectors True for 0-1

(32)

The 0-1 Theorem

0-1 Theorem:

Proof: The non-obvious direction

Assume an identity true for 0-1

Need to prove true for any 0-1 vectors

An identity is true for 0-1 vectors with bitwise OR and AND if and only if

it is true for two valued (0-1) with bitwise OR and AND

(33)

Example:

Theorem 2:

Proof:

0 + 0 ⋅ 0 = 0 0 + 0 ⋅1 = 0 1 + 1⋅ 0 = 1 1 + 1⋅1 = 1

The identity is true for 0-1

Need to prove it for 0-1 vectors

(34)

Theorem 2:

Proof (for 0-1 vectors):

If an identity is not true in general;

then there is an assignment of

elements that violates the equality

Hence, there must be a position in the binary vector that is violated

There exists a 0-1 assignment to the identity that violates the equality, CONTRADICTION!!

By contradiction Assume true for all 0-1 assignments

and not true for some other assignment

Example:

(35)

Recap: The 0-1 Theorem

An identity is true for 0-1 vectors with bitwise OR/AND

if and only if it is true for two valued (0-1) with OR/AND Proof: The easy direction

•  Assume an identity true for 0-1 vectors

•  0-1 is a special case: True for 0-1 The non-obvious direction Q

•  Assume an identity true for 0-1

•  Need to prove true for 0-1 vectors

•  Assume there exists a general ‘identity violating’ assignment

•  Show that there is a 0-1 ‘identity violating’ assignment CONTRADICTION!!

(36)

Operations:

Bitwise AND

Bitwise OR Bitwise Complement

It is a Boolean algebra for any finite n

0-1 vectors of length n, there are 2

ⁿ

vectors Elements:

Application of the the 0-1 Theorem

It is a Boolean algebra with for n=1

By the 0-1 Theorem:

(37)

Boolean Algebra

Two-value? Or not?

(38)

Prove or Disprove

At least one of the following is true:

(39)

xy ^OR(x,y)

00 01 10 11

0 1 1 1

True

for two-valued Boolean algebra

Is it true for 0-1 vectors Boolean algebras?

(40)

+

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

Is it true for 0-1 vectors Boolean algebras?

NO

(41)

If

Then or

Claim is NOT TRUE in general!

The 0-1 Theorem:

True only for identities!!!

+

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

(42)

Examples ‘other’ Boolean Algebras

•  0-1 vectors

•  Arithmetic Boolean algebras

•  Algebra of subsets union / intersection

next next

(43)

Boolean algebra

Boolean integers

^|s|

(44)

Euclid, 300BC

405 = 3x3x3x3x5

297 = 3x3x3x11 27 = 3x3x3

884 = 2x2x13x17

68 = 2 x 2 x 17 612 = 2x2x3x3x17

Greatest Common Divisor

Application: Simplifying fractions

(45)

Euclid, 300BC

405 = 3x3x3x3x5

297 = 3x3x3x11 4455 = 3x3x3x3x5x11

884 = 2x2x13x17

7956 = 2x2x3x3x13x17 612 = 2x2x3x3x17

Least Common Multiple

Application: Adding fractions

(46)

Euclid, 300BC

297x405 = 120,285

gcd(297,405) = 27 lcm(297,405) = 4455

Least Common Multiple Greatest Common Divisor

27x4455 = 120,285

gcd(a,b) x lcm(a,b) = a x b

405 = 3x3x3x3x5 297 = 3x3x3x11 405 = 3x3x3x3x5

297 = 3x3x3x11

(47)

Arithmetic Boolean Algebra

lcm = lowest common multiple gcd = greatest common divisor

The set of elements: {1,2,3,6}

The operations: lcm and gcd

1 is Boolean 0 6 is Boolean 1

(48)

What is the complement?

1 2

3 6

(49)

For which n does it work?

Elements:

Set of all the divisors of an integer n

(50)

complement? The

1 2

4 8

(51)

For which n does it work?

Elements:

Set of all the divisors of an integer n

Prime factors appear at most once in n

(52)

Bunitskiy Algebra 1899

Elements:

The set of divisors of a Boolean integer {1,2,3,5,6,10,15,30}

The 0 and 1 elements: 1 and 30

Boolean Integers 2 x 3 x 5 = 30

2 x 3 x 7 = 42

Every prime in the prime factorization is a power of one (square-free integer)

Euclid, 300BC

(53)

Is Bunitskiy Algebra a Boolean Algebra?

6 = 3x2 3 = 3

2 = 2 1 = 1

11

00 10 01

LCM GCD

11 01

Bitwise OR Bitwise AND

YES

(54)

+

00 01 10 11 00

01 10 11

01 01 11 11

10 11 11 11 10 11 11 11 00

01 10 11

00 01 10 11 00

00 00 00

00 01 00

01

00 00 00 01

10 10 10 11 00

01 10 11

1 2 3 6

1

1 2 3 6

2 2 6 6

3 6 3 6

6 6 6 6 1 2 3 6

1 1 3 3

1 2 1 2

1 1 1 1

1 2 3 6

2 3 6

lcm gcd

Bunitskiy algebra is isomorphic to the algebra of 0-1 vectors,

it is a Boolean algebra!

syntax the same – different semantics

(55)

Boolean algebra

subsets of a set

^|s|

(56)

Algebra of Subsets

S is the set of all points

Elements: all possible subsets of a set S

+ is union of sets:

is intersection of sets

How many elements? 2^|s|

(57)

Example:

Elements:

S =

Operations: union and intersection

Complement:

(58)

Elements:

Corresponding 0-1 vectors:

(10)

(00) (11) (01)

S =

Algebra of subsets is isomorphic to the algebra of 0-1 vectors,

it is a Boolean algebra!

Is the algebra of subsets

a Boolean Algebra? YES

syntax the same – different semantics

(59)

01 11 00 10

00

10

01

11

Elements are: (00), (11), (10), (01)

(60)

01 11 00 10

00

10

01

11

Elements are: (00), (11), (10), (01)

00 10 01 11

10

11 01

10 11 11

11

11 11 11

11 01

Union

Bitwise OR

(61)

Elements are: (00), (11), (10), (01)

01 11 00 10

00

10

01

11

00 00 00 00

00

10 00 10

00

10 01 11

01 01

Intersection

Bitwise AND

(62)

Boolean algebra

an amazing theorem

(63)

Examples of Boolean Algebras

•  0-1 (two valued) Boolean algebra OR / AND

•  0-1 vectors

bitwise OR / bitwise AND

• Arithmetic Boolean algebras lcm / gcd

•  Algebra of subsets union / intersection

Size 2

^k

They are isomorphic!

(64)

Representation Theorem (Stone 1936):

Every finite Boolean algebra is isomorphic to a Boolean algebra of 0-1 vectors

Algebra 1 Algebra 2

elements elements

operations operations

(65)

Representation Theorem (Stone 1936):

Every finite Boolean algebra is isomorphic to a Boolean algebra with elements being bit vectors of finite length with bitwise operations OR and AND

Two Boolean algebras with m elements are isomorphic

Provides intuition beyond the axioms:

We can ‘naturally’ reason about results in Boolean algebra Every Boolean algebra has 2 elements ^k

(66)

Marshall Stone

Marshall Stone 1903-1989

Proved in 1936

90AB = years After Boole

The Boolean Syntax invented in 1847 has a unique representative semantic!!!

Harlan Fiske Stone

12^th Chief Justice of the US 1941-1946

Marshall entered Harvard in 1919 intending to continue his studies at Harvard law school; fell in love with Mathematics, and the rest is history…

Marshall had a passion for travel. He began traveling when he was young and was

on a trip to India when he died....

(67)

Quiz #3 – 10min

What is the complement of

Use the following DeMorgan Laws:

And the axioms:

Show

your work and

justify

every step!

Solution in this form:

(68)

What is the complement of

DeMorgan Laws:

And the axioms:

DeMorgan

A4 A4 A3

A2 A3

A1

(69)

One week!

(70)

(71)

2 symbol adder c

s

d1 d2

c

parity

majority

(72)

(73)

(74)

Need to provide a complete proof No duality arguments

For (c): Sum of products (no need to expand to DNF) See the solution to Quiz #3

(75)

1 2

3 4

Prove 1 Prove 2

multiply