Algorithms: A Brief Tour

(1)

Algorithms: A Brief Tour

Mario A. López

Department of Mathematics Department of Computer Science

University of Denver

2

“Two ideas lie gleaming in the scientist’s chest: the first is the calculus, the second, the

algorithm. The calculus made modern science possible; but it has been the algorithm that has made possible the modern world.”

David Berlinski, May 2000.

(2)

3

Selected Applications

• Human genome:

– 100,000 genes, 3 billion base pairs.

– Tools for analysis and search.

• Management of the Internet

– Routing.

– Search engines (e.g., Google).

• Electronic commerce

– privacy and security (public key cryptography, electronic signatures).

– data mining.

• Geographic information systems

– Shortest route (mapquest).

– Route inspection in road networks.

• Manufacturing

– Shortest cycle through a list of locations.

– Constraint satisfaction systems

• Resource management

– Electoral campaigns, crew assignment, scheduling

Issues

• What are algorithms?

• How do we describe algorithms?

– Models of computation

• What makes good algorithms?

– Correctness, complexity models

• How can we design good algorithms?

– Design patterns

• What problems can be solved by algorithms?

– Computability and tractability

(3)

What are algorithms?

6

What is an Algorithm?

• Finite and effective procedure that takes one or more values as input and produces one or more values as output in finite time.

– Instructions expressed in the language of the processor – A partial function f : N → N

– Solves a general problem Examples:

– Find the greatest common divisor (gcd) of two positive integers.

– Sort a list of numbers.

– Multiply two matrices.

– Find the shortest route that visits all sites in a set of points.

(4)

7

Algorithms as Functions

• An algorithm computes a partial function

f : N→→→→N from non-negative integers (the input) to non-negative integers (the output)

Example: Convert string from lower to upper case

f (“ace”) = “ACE”

“ace” f “ACE”

f

1100001100100011001100101 1000001001000011001000101

25577061 f 17073733

Problem Characterization

A general problem is specified by

1. (Infinite) collection of input instances 2. Properties that output must satisfy

Example: Sorting

Input: list of values 〈x₁, x₂,…, x_n〉 from ordered universe (U, <)

Output: permutation π such that x_π(i)≤ x_π(i+1)

〈7, 3, 1, 4〉

〈Tina, Beto, Ana, Luis〉

〈1, 3, 4, 7〉

〈Ana, Beto, Luis, Tina〉



 →

³²⁴¹

〈(5,1), (0,5), (2,2), (6,7)〉 〈(2,2), (0,5), (6,7), (5,1)〉

(5)

9

Euclid’s Algorithm (c. 300 B.C.)

• Finds the greatest common divisor of two positive integers a, b.

Example: gcd(252,105) = 21

• Applications:

– RSA encryption (electronic commerce) – Generation of musical rhythms

– Finding roots of polynomials

– Unlimited precision computer arithmetic.

252−105 = 147 ⇒ gcd(252,105)= gcd(147,105) 147−105 = 42 ⇒ gcd(147,105)= gcd(105,42)

10

Euclid’s Version

• How many iterations are required?

gcd(a, b) repeat

if b > a then

t ← a; a ← b; b ← t r ← a −b; a ← b; b ← r until b = 0

return a

a b r

252 105 147

147 105 42

105 42 63

63 42 21

42 21 21

21 21 0

a b r

3978 1590 2388

2388 1590 798

1590 798 792

798 792 6

… … …

(6)

11

An Improved Version

gcd(a, b)

while b ≠ 0 do r ← a mod b a ← b

b ← r return a

• Why is this correct?

a b r

252 105 42

105 42 21

42 21 0

a b r

3978 1590 798

1590 798 792

798 792 6

792 6 0

Claim. a ≥b > 0 ⇒ gcd(a,b) = gcd(b, a mod b)

Proof. Argue that d |a and d |b iff d |b and d | (a mod b)

• How many iterations are necessary and sufficient?

12

Analysis

gcd(a, b)

while b ≠ 0 do r ← a mod b a ← b

b ← r return a

How do you measure efficiency?

– time, e.g., seconds – some other variable

proportional to time – As a function of input size

Theorem. a > b ⇒ r < a/2

1. b ≤ a/2 ⇒ r < b ≤ a/2 2. b > a/2 ⇒ r = a – b < a/2

• This second implementation of Euclid’s algorithm takes time proportional to # of input digits

(7)

How do we describe algorithms?

14

Models of Computation

• Many different models: Turing machines, real RAM, PRAM, etc.

• Two components

– data (e.g., natural numbers {0, 1, 2, …}) – instructions (e.g., reset (to 0), increment,

iterate a fixed number of times)

t ← 0 do x

y ← 0 do t

y ← y + 1 t ← t + 1 y ← 0

do x

y ← y + 1

What is y = f(x) ?

(8)

15

Data

• Atomic types: variables and constants

– Variables can store a single value which can

change one or more times during execution

– Variables have a type (e.g., integer, real, boolean)

• Compound types

– Arrays

Example: A(2)=3.7

– Structures

Example: B.owner = “Perez”

3 .

1 −23.7 8.0 9.1

0 1 2 8 9

A

grass Perez 123.9 land type owner area

B

16

Linked Structures

2 .

1 2.5 4.1 4.2 9.3

1.2 3

4.2 4

2.5 5

9.3 0

4.1 2

1 2 3 4 5

6

2 8

4 9

1

(9)

17

Instructions

• Assignment

• Control Structures

– Conditional: if <condition> then …

– Iteration:

• while <condition> do …

• repeat … until <condition>

• for <id> ← <initial> to <final> do…

a ac b

b z z

x

x← −(3 /2−1) ←− + ²−4 /2

1 1

x≥ y+ then x← if

1

20 1

i← to do S ←S+ for

18

Control Structures…

• Recursion

– A function is recursive if it calls itself – Must include a base case

function gcd(a, b)

if b = 0 then return a else

c ← a mod b

return gcd(b, c) end

(10)

19

Example: Insertion Sort

) ( ) ( 1

, of n permutatio

:

, ,

:

j i

n

A A

j i A A

A A

π

π < ⇒ π ≤

= Output

Input K

[ ]

[ ] [ ] [ ] [ ]

A k i

A

i i

i A i

A

k i A i

j i

j A k

n j

n A

. 8

1 . 7

1

. 6

1 5.

0

. 4

1

. 3

. 2

2

. 1

) , ( Sort

return

do and

while

do to for

←

+ ←−+>−← >

←←←

[ ] [ ] [ ] [ ] [ ]

A k i

A

i i

i A i

A

k i A i

n A k

n A n

n A

. 8

1 . 7

1

. 6

1 5.

0

. 4

. 3

) 1 , ( Sort . 2

1

. 1

) , ( Sort

return

do and

while then if

←

+←+>−← >

← −

>

i j

k

<k

What makes good algorithms?

(11)

Which Algorithm Would You Use?

Sort(A, p, q)

1. if A[p] > A[q] then

2. exchange A[p] with A[q]

3. if p + 1 ≥ q then return 4. t ←(q − p + 1) / 3 5. Sort(A, p, q − t) 6. Sort(A, p + t, q) 7. Sort(A, p, q − t)

end Sort(A, n)

1. for j ← 2 to n do 2. k ← A[j]

3. i ← j − 1

4. while i > 0 and A[i] > k do 5. A[i + 1] ← A[i]

6. i ← i − 1 7. A[i+1] ← k end

Properties of a good algorithm

• Correctness

– Algorithm must implement the correct input to output transformation

• Efficiency

– Time and space complexities

– Express as a function of input size and output size

• Implementation complexity

22

(12)

23

Algorithm Complexity

• To understand performance of algorithm A, it is not enough to run it on one input.

• Need to know behavior (memory, running time) over all possible input instances.

Example: Running time

• Minimum

• Maximum

• Average

• Complexity is usually expressed as a function (e.g., a polynomial) of the input size n

Example: Sorting

• For every instance I, run A and plot point (|I|, T_A(|I|) )

24

(13)

Worst, Best, and Average

• The worst case complexity is the function defined by the maximum number of steps taken on any instance of size n.

• The best case complexity is the function defined by the minimum number of steps taken on any instance of size n.

• The average-case complexity is the function defined by an average number of steps taken on any instance of size n.

Each of these complexities defines a numerical function:

time vs. size!

25

26

Insertion Sort

[ ]

[ ] [ ] [ ]

[ ]

1 . 7

1 .

6

1

5.

0 . 4

1 . 3

. 2

2 . 1

) , ( Sort

k i

A

i i

i A i

A

k i A i

j i

j A k

n j

n A

← + ← +

←

+ > >

−

← ←

←

do and

while

do to

for

) 1 (

) 1 ( ) 1 (

) 1 (

7 6 2 5 2 4 2 3 2 1

−

⋅

−

⋅

−

⋅

−

⋅

−

⋅

∑

=

n c

t c

n c

n

j j

n

j j

n

j j

Cost

c t b

an n

T

ⁿ

j j

+

⋅ +

= ∑

=2

) (

i j

k

<k

(14)

Insertion Sort: Analysis

• Worst, best, and average depend on the values t

_j

.

– Best case: t_j= 1 ⇒

T(n) = an + b(n − 1) + c = a₁n + a₀ – Worst case: t_j= j ⇒

T(n) = an + b(n+2)(n − 1)/2 + c = b₂n²+ b₁n + b₀ – Average case: t_j= j / 2

T(n) = an + b(n+2)(n − 1)/4 + c = d₂n²+ d₁n + d₀

27

Exact Analysis is Difficult

• Exact values of constants (a

_i

, b

_i

, d

_i

, …) depend on:

– machine – compiler

– implementation of algorithm

⇒ Analysis not very general

28

• Best, worst, and average case are difficult to

deal with precisely because too many details

(15)

A Simpler Approach

• It is easier to talk about upper and lower bounds of the function in a manner that avoids machine and implementation details

– Ignore machine dependent constants

– Drop lower order terms, e.g., n³+ 27n − 3 ≈ n³for large n – Look at growth rate as n → ∞

29

Names of Bounding Functions

30

• f(n)=O(g(n)) means c⋅g(n) is upper bound for f(n)

• f(n)=Ω(g(n)) means c⋅g(n) is lower bound for f(n)

• f(n)=Θ(g(n)) means c₁⋅g(n) is upper bound for f(n) and c₂⋅g(n) is lower bound for f(n)

c, c₁, and c₂are constants independent of n bounds hold for “sufficiently large” n

(16)

Example: Insertion Sort

• Worst case grows as n

²

⇒ T

_max

(n) = Θ(n

²

)

• Best case grows as n ⇒ T

_min

(n) = Θ(n)

• Average case grows as n

²

⇒ T

_ave

(n) = Θ(n

²

)

• Also, insertion sort takes between n and n

² T(n) = Ω(n) and T(n) = O(n²)

31

2 3 2

)

1

( :

Time T n a n a

ⁿ

t a

j j

+

⋅ +

= ∑

=

(

₁ ₂

)

₃ ₁ ₂ ₃

2 ) 1 )(

2 ) (

(

1 n n a

a n a n T a

n a

a + − +

⋅ +

≤

− + +

Asymptotic Notation: O

• Captures the idea of upper bound for T(n)

32 0

0: ( ) ( )

, ))

( ( )

(n O f n c n T n c f n n n

T = ⇔ ∃ ≤ ⋅ ∀ ≥

) (n f c ⋅

) (n T

n0

Time

n

(17)

Examples

• Which of the following is true?

33

) (

) ( 2 4

5

) ( 2 4 10

) ( 6 100

2

3 2

3

3 2

n O n

n

n O n

n

n O n

=

− +

= + +

= +

34

Asymptotic Notation: Ω

0

: ( ) ( )

, ))

( ( )

( n f n c n T n c f n n n

T = Ω ⇔ ∃ ≥ ⋅ ∀ ≥

) (n f c ⋅

) (n T

n0

• Idea of lower bound for T(n)

n

Time

(18)

Examples

35

• Which of the following is true?

) (log

) ( 3

5 . 0

) ( 6

100 n n

n n

Ω

=

Ω

=

− + = Ω

Asymptotic Notation: Θ

• Idea of equivalent bound for T(n)

)) ( ( ) (

&

)) ( ( ) (

n f n

T n

f n

T n f n

T

Ω

= Ο

⇔= Θ

=

)

2 f(n c ⋅

)

1 f(n c ⋅

) (n T

n0 n

Time

(19)

More Examples: O, Ω, Θ

• Think of = as meaning “in the set of functions” 37

38

Common Running Times

(20)

39

Graphs

• A graph G=(V,E) consists of:

– set V of n vertices

– set E of m edges (directed or not)

• Problems

– Find the minimum number of colors that suffices to color the map, – Find a shortest cycle or a path that visits all capital cities.

40

Two Related Problems

Given a graph G=(V, E)

1. Find a route that visits each edge exactly once.

2. Find a route that visits each vertex exactly once.

HNL

SFO

LAX

ORD

DFW

LGA

PVD

MIA 849

1120

(21)

41

Rendering Triangular Meshes

• Triangular meshes are the most common model used for representing 3D objects in computer graphics.

• Hardware optimized for shading and rendering one triangle at a time

• Bottleneck resides in transferring the geometric data to the GPU

42

OpenGL Triangle Strips

• Goal: specify most triangles with just one additional vertex

Point3d strip[n];

glBegin(GL_TRIANGLE_STRIP);

for(int i=0; i<n; i++)

glVertex2fv(strip[i]);

glEnd();

1 2

3 4

5 6 7

(22)

43

Goal

• Render model using one triangle strip.

– Can this always be done?

• Partition model into the smallest possible number of triangle strips?

– Can this always be done?

Robot Tour Optimization

44

• Robot arm equipped with a tool, e.g., soldering iron.

• Construct an ordering of the contact points, so the robot visits (and solders) the points in order.

• We seek the order which

minimizes the time it takes to

assemble the circuit board.

(23)

Nearest Neighbor Tour

Pick and visit an initial point p

₀

i = 0

While there are still unvisited points i = i + 1

Let p

_i

be the closest unvisited point to p

_i−1

Visit p

_i

Return to p

₀

from p

_i ₄₅

• Start at some point p₀and then walks to its nearest neighbor p₁first, then repeats from p₁, etc. until done.

Nearest Neighbor Example

46 0

1

2 7

3 8

6

5

4

(24)

Nearest Neighbor is Wrong!

47 0 1

2 3

4

6 5

3 4

2 5

0 1 6

Closest Pair Tour

48

• How do you do this? Need to detect cycles, 3-way branches

3 4

2 5

0 1 6

• Repeatedly connect the closest pair of points whose connection will not cause a cycle of length < n or a 3-way branch, until all points are in one tour.

(25)

Closest Pair Tour

Sort the (implied) edges E in ascending order C = ∅ (initialize tour)

For each edge e in sorted order

if e does not create a vertex of degree 3 or cycle

add e to C

if |C| = n − 1 then

close and return C

49

• Keep chains of vertices. Only endpoints of distinct chains can be connected.

• Initially, each vertex is in its own chain.

Closest Pair is Wrong!

50

1+ε 1+ε

1−ε

Optimal tour

(26)

A Correct Algorithm: Exhaustive Search

d = ∞

for each permutation π

_i

of the input points if length( π

_i

) < d

d = length( π

_i

) Min = π

_i

Return Min

51

• Try all possible orderings of the points, then select the one which minimizes the total length.

Exhaustive Search is Slow!

No algorithm that is both efficient and correct is known

One of 7 millennium problems (Clay Math Institute)

• Not practical when n exceeds 15-20 points 20! = 2,432,902,008,176,640,000

If can evaluate 100 permutations/µsec then will need

> 630 years to find answer

• Later, through dynamic programming, we will reduce the time to Θ(n²2ⁿ)

(27)

Why not use a supercomputer?

• For sufficiently large instances, a faster algorithm running on a slow computer will always outperform a slower algorithm on a fast computer.

• Usually, problems don’t have to get that large before the faster algorithm wins.

53

Intractability

• Not all problems that can be solved in principle can be solved in practice

• A problem is tractable if it can be solved in polynomial time

– OK, but is Ω( n¹⁰⁰⁰) practical?

What choice do we have left?

• An approximation algorithm is an “incorrect”

algorithm whose answer is (ideally) guaranteed to be within a certain bound of the correct answer.

• Often based on heuristics

• Approximation algorithms usually run very fast

• Often there is a trade-off between speed and accuracy.

54

(28)

Minimum Spanning Tree Heuristic

55

• Simple approximation algorithm

1. Compute MST on complete graph

2. Double each edge {u,v} to appear in both directions (u,v) and (v,u).

3. Walk around the resulting tour short-circuiting redundant vertices.

56

Example: MST Heuristic for TSP

(a) Input set of points (b) Walk around twice with shortcuts

(29)

57

2-opt Local Search Heuristic

a) Original tour

b) New tour after a 2-change

58

Example: 2-opt Heuristic

• 2-changes from an initial NN tour

(30)

59

How about 3-changes?

(a) Initial tour (b-c) two possible 3-changes

(31)

How do we design good algorithms?

Design Paradigms

• Incremental

• Divide-and-conquer

• Dynamic programming

• Greedy

• Randomization

62

(32)

Incremental

1. Base case. Solve 〈a₁, a₂,… a_c〉 for small constant c 2. Inductive step. Extend solution for 〈a₁,a₂,…,a_i-1〉

to solution for 〈a₁,a₂,…,a_i-1,a_i〉

63

[ ]

[ ] [ ][ ] [ ]

A z i

A

i i

i A i

A

z i A i

j i

j A z

n j

n A

. 8

1 . 7

1

. 6

1 5.

0

. 4

1

. 3

. 2

2

. 1

) , ( Sort

return

do and

while

do to for

←

+←−+>+← >

←←←

[ ]

z j A z

z j A

n j

A z

n A

. 5

] [ . 4

. 3

2

. 2

] 1 [ . 1

) , ( Max

return

then if

do to for

←>

←

64

Convex Hull

i i

n

P H

p p

P

P P

H

p p

P

to solution the

, and } , , { Let

of hull convex the

), conv(

find

plane, the

in points of

} , , { set

a Given

1

K

= = =

} conv{

4

2)

triangle the

i.e., , ) , , conv(

Let 1)

1

3 2 1 3

2 1 3

i i

i H p

H

n i

p p p p

p p H

∪

←←=

−

do to for

(33)

Application: Character Smoothing

Input: polygon with many vertices

Output: similar polygon with much fewer vertices

65

66

pi

ql

qr

H

Incremental Algorithm 1

i l

r

r l

j

j j i j

j i

i i

n

p H q

q

q q

q

q q p q

q p

n j

H p

q q H n

i

p p p H

i

by in , , replace 7)

left}

if right, if { point tangency a

is 6)

) , , turn(

) , , ( turn 5)

points}

tangency {find

1

4) 3)

} , , {let

4 2)

) , , conv(

1)

1 1

1

1 3

2 1

1

− +

+

−

=

←

∉

=

←

−

K

then if

do to for

then if

do to for

) ( : Time Θ n²

(34)

67

Incremental Algorithm 2

i l

r

r i

l i

n

p H q

q

q p

q q H n

i

p p p H

x P

i

by in

, , replace

6)

find to clockwise counter

move from

Starting

5)

find to clockwise move

from Starting

4)

} , , {let

4

3)

) , , conv(

2)

coordinate -

by Sort 1)

1 1

1 3

2 1

1

− +

−

= −

←

K do K to for

pi

ql

qr

H

^pⁱ⁻¹

Time is dominated by sorting! why?

Divide-and-Conquer

68

1. Divide by splitting problem into 2 or more smaller sub-problems (e.g., 〈a

₁

,…, a

_n/2

〉 and

〈a

_n/2+1

,…, a

_n

〉)

2. Conquer by solving sub-problems independently

3. Combine partial solutions into a solution to the original instance

Example: if divide the problem into a instances of size n / b each, then the total time is

) ( ) ( ) / ( )

(n aT n b d n c n

T = ⋅ + +

(35)

DAC With 2 Sub-problems

69

divide

conquer

combine

Incremental as a case of DAC

70

(36)

71

Example: Convex Hull

• Given a set S ={p

₁

,…,p

_n

} of points on the plane find conv(S)

1. Sort S lexicographically

2.

Divide

: S

₁

={p

₁

,…,p

_n/2

} and S

₂

={p

_n/2+1

,…,p

_n

} 3.

Conquer

: let P

₁

= conv(S

₁

) and P

₂

= conv(S

₂

) 4.

Combine

: P = conv(P

₁

,P

₂

), but how?

72

Convex Hull: Combine Step

• There are four tangents

• Need to find upper and lower tangents

(37)

73

Finding the Lower Tangent

1. a ← rightmost point of left polygon A 2. b ← leftmost point of right polygon B

3. while T = ab not lower tangent to A and B do 4. while T not lower tangent to A do

5. a ← a – 1

6. while T not lower tangent to B do

7. b ← b + 1

74

Finding the Lower Tangent…

A

B a

b

(38)

75

Merge Sort

MergeSort(A, p, r)

if p = r then return else

q ← (p+r)/2

MergeSort(A, p, q) MergeSort(A, q+1, r) Merge(A,p,q,r)

end

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

• • • • • • • • •

Merging

• Keep track of smallest element in each sorted half.

• Insert smallest of two elements into auxiliary array.

• Repeat until done.

•A

(39)

•77

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A ••G • • • • • • • •

Merging

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A •G ••H • • • • • • •

Merging

(40)

•79

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H ••I • • • • • •

Merging

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I ••L • • • • •

Merging

(41)

•81

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I ••L • • • • •

Merging

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I •L ••M • • • •

Merging

(42)

•83

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I •L •M ••O • • •

Merging

auxiliary array

smallest smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I •L •M •O ••R • •

Merging

(43)

•85

auxiliary array first half

exhausted smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I •L •M •O •R ••S •

Merging

exhausted smallest

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I •L •M •O •R •S ••T

Merging

(44)

•87

exhausted

second half exhausted

•A •G •L •O •R •H •I •M •S •T

•A •G •H •I •L •M •O •R •S •T

Merging

88

(45)

Running Time of Merge Sort and Convex Hull

• How long does the combine step (merge, find tangents) take?

• Time taken can be described by a recurrence:

89

T(n) = 2T(n /2) + cn

• What does this resolve to?

Recursion Tree for T(n) = 2T(n /2)+cn

90

T(n) = 2T(n /2) + cn = Θ( n log n)

(46)

91

Running time comparison

Input size, n Θ(n²) Θ(n log n)

10 10 µsec 332 µsec

100 1 msec 6.64 msec

1,000 100 msec 100 msec

10,000 10 sec 1.3 sec

100,000 17 min 16 sec

1,000,000 28 hours 3 min

10,000,000 116 days 39 min

T₁(n) = 0.1 n² T₂(n) = 10 n log n

Can we expect to do better?

• How fast can we sort?

– Answer depends on computational model – Concentrate on comparison sorts

• Basic operation is the comparison of two elements

• Only use comparisons to determine relative order

• Only count comparisons to determine complexity

• Insertion sort takes Θ(n

²

)

• Merge sort takes Θ(n log n)

• Can we do better than Θ(n log n)?

(47)

Comparison Sort Model

• Basic operation is the comparison of two elements

– Only use comparisons to determine relative order – Only count comparisons to determine complexity

• Lower bounds

– Will show Ω(n log n) lower bound for this model

2

94

Decision Trees

• A decision tree is an abstraction of a comparison sort

• It represents the set of all possible

comparisons made by a fixed algorithm on inputs of a fixed size

• Abstracts away everything else, such as control and data movement

• Only comparisons are counted

(48)

95

Example

• Insertion sort, n = 3, e.g., sort 〈3,5,2〉

• Each node labeled with original element indices

• Each leaf labeled by permutation found by algorithm

• Path in bold corresponds to a₃≤ a₁≤ a₂

More generally…

• Want to sort 〈a₁,…,a_n〉

• Each internal node has label a_i:a_i, where i,j∈{1,…n}

• Left subtree contains later comparisons when i ≤ j

• Right subtree contains later comparisons when i > j

• Each leaf node has a permutation of 〈1,…,n〉 that corresponds to the correct sorted order of the input

96

(49)

97

Properties

For a particular algorithm

– One tree for each n

– All possible execution traces are represented

• A specific run ⇒ a path from root to leaf

• How many leaves does a decision tree have?

• What is the length of longest path from root to leaf?

Depends on the algorithm!

– insertion sort?

– merge sort?

98

Lower Bound for Sorting

Theorem. Any decision tree for sorting n

elements has height Ω(n log n)

Proof.

A binary tree of height h has ≤ 2^hleaves

Decision tree for correct algorithm has ≥ n! leaves

⇒ n! ≤ #leaves ≤ 2^h⇒h ≥ logn!

Corollary. Merge sort is asymptotically optimal

(50)

Maximize Stock Trading Profit

Goal: when should you buy and sell if you want to maximize your profit?

99

Proposed Algorithms

• Algorithm 1. Buy at the lowest point and sell at the highest point after it.

• Algorithm 2. Sell at the highest point and buy at the lowest point before it.

• Algorithm 3. Choose the best of 1 or 2.

• Algorithm 4. Consider all pairs (i,j) of days where j > i and choose the best pair.

(51)

A Transformation

• What if you work instead with the sequence of daily changes?

Goal: find a contiguous subarray whose value has a largest sum

101

Maximum Subarray

• Algorithm 1. For each subarray A[i..j] find the net change and keep the maximum.

102 102

MaxSubSum1(A, n) 1 best ← 0

2 for i ← 1 to n do 3

for j ← i to n do

4 sum ← 0

5

for k ← i to j do

6 sum ← sum + A[k]

7

if sum > best then

8 best ← sum

9 return best

end

(52)

Algorithm 2

103

MaxSubSum2(A, n) 1 best ← 0

2 for i ← 1 to n do 3 sum ← 0

4

for j ← i to n do

5 sum ← sum + A[j]

6

if sum > best then

7 best ← sum

8 return best

end

Algorithm 3: DAC

(53)

105

MaxSubSum(A, i, j) 1 if i = j then

2 if A[i] > 0 then return A[i] else return 0 3 m ← (i + j) / 2

4 maxL ← MaxSubSum(A, i,m) 5 maxR ← MaxSubSum(A, m+1, j) 6 tmpL ← maxtmpL ← 0

7 for k ← m downto i do 8 tmpL ← tmpL + A[k]

9 if tmpL > maxtmpL then maxtmpL ← tmpL 10 tmpR ← maxtmpR ← 0

11 for k ← m+1 to j do 12 tmpR ← tmpR + A[k]

13 if tmpR > maxtmpR then maxtmpr ← tmpR 14 return max{maxL, maxR, maxtmpL+maxtmpR}

end

Time: T(n) = 2T(n/2)+Θ(n) = Θ(n log n)

Closest Pair

• Given a set S =

{p₁, p₂,…,p_n}

of points on the plane find a and b such that

• Brute force takes Θ(n

²

) time

• Can we do better?

n j i p

p p

p_a, _b)≤ dist( _i, _j),∀1≤ ≠ ≤ dist(

106

(54)

107

To compute CP(S):

1. Sort S lexicographically, i.e., such that

2.

Divide

: and 3.

Conquer

: let and

4.

Combine

: how? is the answer?

A Divide and Conquer Solution

) , min( ₁ ₂

= δ δ

δ ) ( ₁

1 =CP S

δ δ₂ =CP(S₂)

{

₁ _/₂

}

1 p , ,p_n

S = K S₂

{

p_n_/₂ ₁,K,p_n

}

= + 1 1

1 + +

+ ∨ = ∧ ≤

< _i _i _i _i _i

i x x x y y

x

108

Closest Pair: Combine Step

→

←

^δ

← →

^δ

δ

(55)

How many points can you have in the box?

109

δ δ

δ

Coincident points, one in S₁and one in S₂

S₁

S₂

Matrix Multiplication

Given two n × n matrices A and B find C=A×B

Time?

^T⁽ⁿ⁾⁼ ^Θ

( )

ⁿ³ ^or^T⁽^N⁾⁼ ^Θ

(

^N ^N

)

110