# Even More Dynamic Programming

(1)

## Dynamic Programming

(2)

### Edit Distance

More formally:

The edit distance between two strings is:

The minimum number of deletions, insertions, and substitutions to transform string π₯ into string π¦.

Deletion: removing one character

Insertion: inserting one character (at any point in the string) Substitution: replacing one character with one other.

(3)

### Example

B A B Y Y O D A S

sub sub ins sub del

T A S T Y S O D A

Whatβs the distance between babyyodas and tastysoda?

Quick Checks β can you explain these?

If π₯ has length π and π¦ has length π, the edit distance is at most max(π₯, π¦)

The distance from π₯ to π¦ is the same as from π¦ to π₯ (i.e. transforming π₯ to π¦ and π¦ to π₯ are the same)

Distance: 5, one point for each colored box

(4)

### Finding a recurrence

What information would let us simplify the problem?

What would let us βtake one stepβ toward the solution?

βHandlingβ one character of π₯ or π¦

i.e. choosing one of insert, delete, or substitution and increasing the

βdistanceβ by 1

πππ(π, π) is the edit distance of the strings π₯1π₯2 β― π₯π and π¦1π¦2 β― π¦π. (weβre indexing strings from 1, itβll make things a little prettier).

(5)

### The recurrence

βHandlingβ one character of π₯ or π¦

i.e. choosing one of insert, delete, or substitution and increasing the

βdistanceβ by 1

What does delete look like? πππ(π β 1, π) (delete character from π₯ match the rest)

Insert πππ(π, π β 1) Substitution: πππ(π β 1, π β 1)

Matching charcters? Also πππ(π β 1, π β 1) but only if π₯π = π¦π

(6)

### The recurrence

βHandlingβ one character of π₯ or π¦

i.e. choosing one of insert, delete, or substitution and increasing the

βdistanceβ by 1

πππ π, π = ΰ΅

min 1 + πππ π β 1, π , 1 + πππ π, π β 1 , π[π₯π β  π¦π] + πππ π β 1, π β 1

π if π = 0

π if π = 0

Delete Insert Sub and matching

βIndicatorβ β math for βcast

bool to intβ

(7)

### Edit Distance

πππ(π, π)0 B, 1 A, 2 B, 3 Y, 4 Y, 5 O, 6 D, 7 A, 8 S, 9

0

T 1 A 2 S 3 T 4 Y 5 S 6 O 7 D 8 A 9

(8)

### Edit Distance

πππ(π, π)0 B, 1 A, 2 B, 3 Y, 4 Y, 5 O, 6 D, 7 A, 8 S, 9

0 0 1 2 3 4 5 6 7 8 9

T 1 1 A 2 2 S 3 3 T 4 4 Y 5 5 S 6 6 O 7 7 D 8 8 A 9 9

(9)

### Edit Distance

πππ(π, π)0 B, 1 A, 2 B, 3 Y, 4 Y, 5 O, 6 D, 7 A, 8 S, 9

0 0 1 2 3 4 5 6 7 8 9

T 1 1 1

A 2 2 S 3 3 T 4 4 Y 5 5 S 6 6 O 7 7 D 8 8 A 9 9

1 + 1 ππππ‘, πππππ‘π 1 + 1 π’π, πππ πππ‘ 0 + 1(ππππ, π π’π)

(10)

### Edit Distance

πππ(π, π)0 B, 1 A, 2 B, 3 Y, 4 Y, 5 O, 6 D, 7 A, 8 S, 9

0 0 1 2 3 4 5 6 7 8 9

T 1 1 1 2 3 4 5 6 7 8 9

A 2 2 2 1

S 3 3 T 4 4 Y 5 5 S 6 6 O 7 7 D 8 8 A 9 9

2 + 1 ππππ‘, πππππ‘π 2 + 1 π’π, πππ πππ‘

1 + 0(ππππ, πππ‘πβ)

(11)

### Edit Distance

πππ(π, π)0 B, 1 A, 2 B, 3 Y, 4 Y, 5 O, 6 D, 7 A, 8 S, 9

0 0 1 2 3 4 5 6 7 8 9

T 1 1 1 2 3 4 5 6 7 8 9

A 2 2 2 1 2 3 4 5 6 7 8

S 3 3 3 2 2 3 4 5 6 7 7

T 4 4 4 3 3 3 4 5 6 7 8

Y 5 5 5 4 4 3 3 4 5 6 7

S 6 6 6 5 5 4 4 4 5 6 6

O 7 7 7 6 6 5 5 4 5 6 7

D 8 8 8 7 7 6 6 5 4 5 6

A 9 9 9 8 8 7 7 6 6 4 5

(12)

### Edit Distance

πππ(π, π)0 B, 1 A, 2 B, 3 Y, 4 Y, 5 O, 6 D, 7 A, 8 S, 9

0 0 1 2 3 4 5 6 7 8 9

T 1 1 1 2 3 4 5 6 7 8 9

A 2 2 2 1 2 3 4 5 6 7 8

S 3 3 3 2 2 3 4 5 6 7 7

T 4 4 4 3 3 3 4 5 6 7 8

Y 5 5 5 4 4 3 3 4 5 6 7

S 6 6 6 5 5 4 4 4 5 6 6

O 7 7 7 6 6 5 5 4 5 6 7

D 8 8 8 7 7 6 6 5 4 5 6

A 9 9 9 8 8 7 7 6 6 4 5

(13)

### Edit Distance

πππ(π, π)0 B, 1 A, 2 B, 3 Y, 4 Y, 5 O, 6 D, 7 A, 8 S, 9

0 0 1 2 3 4 5 6 7 8 9

T 1 1 1 2 3 4 5 6 7 8 9

A 2 2 2 1 2 3 4 5 6 7 8

S 3 3 3 2 2 3 4 5 6 7 7

T 4 4 4 3 3 3 4 5 6 7 8

Y 5 5 5 4 4 3 3 4 5 6 7

S 6 6 6 5 5 4 4 4 5 6 6

O 7 7 7 6 6 5 5 4 5 6 7

D 8 8 8 7 7 6 6 5 4 5 6

A 9 9 9 8 8 7 7 6 6 4 5

(14)

### Dynamic Programming Process

1. Define the object youβre looking for

2. Write a recurrence to say how to find it

3. Design a memoization structure

4. Write an iterative algorithm

Minimum Edit Distance between π₯ and π¦

β

π Γ π Array

Outer loop: increasing rows (starting from 1) Inner loop: increasing column (starting from 1)

(15)

(16)

### Maximum Contiguous Subarray Sum

We saw an π(π log π) divide and conquer algorithm.

Can we do better with DP?

Given: Array π΄[]

Output: π, π such that π΄ π + π΄ π + 1 + β― + π΄[π] is maximized.

(17)

### Dynamic Programming Process

1. Define the object youβre looking for

2. Write a recurrence to say how to find it

3. Design a memoization structure

4. Write an iterative algorithm

(18)

### Maximum Contiguous Subarray Sum

We saw an π(π log π) divide and conquer algorithm.

Can we do better with DP?

Given: Array π΄[]

Output: π, π such that π΄ π + π΄ π + 1 + β― + π΄[π] is maximized.

For today: just output the valueπ΄ π + π΄ π + 1 + β― + π΄[π].

Is it enough to know OPT(i)?

(19)

### Trying to Recurse

0 1 2 3 4 5 6 7

5 -6 3 4 -5 2 2 4

πππ 3 would give π =2, π = 3

πππ(4) would give π = 2, π = 3 too

πππ(7) would give π = 2, π = 7 β we need to suddenly backfill with a bunch of elements that werenβt optimalβ¦

How do we make a decision on index 7? What information do we need?

(20)

### What do we need for recursion?

If index π IS going to be included

We need the best subarray that includes index π β π

If we include anything to the left, weβll definitely include index π β 1 (because of the contiguous requirement)

If index π isnβt included

We need the best subarray up to π β 1, regardless of whether π β 1 is included.

(21)

### Two Values

Need two recursive values:

πΌππΆπΏππ·πΈ(π): sum of the maximum sum subarray among elements from 0 to π that includes index π in the sum

πππ(π): sum of the maximum sum subarray among elements 0 to π (that might or might not include π)

How can you calculate these values? Try to write recurrence(s), then think about memoization and running time.

Pollev.com/robbie

(22)

### Recurrences

πΌππΆπΏππ·πΈ π = α max π΄ π , π΄ π + πΌππΏπΆππ·πΈ π β 1 if π β₯ 0

0 otherwise

πππ π = α max πΌππΆπΏππ·πΈ π , πππ π β 1 if π β₯ 0

0 otherwise

If we include π, the subarray must be either just π or also include π β 1.

Overall, we might or might not include π. If we donβt include π, we only have

access to elements π β 1 and before. If we do, we want πΌππΆπΏππ·πΈ(π) by definition.

(23)

### Example

0 1 2 3 4 5 6 7

5 -6 3 4 -5 2 2 4

π΄

0 1 2 3 4 5 6 7

πππ(π) 5

0 1 2 3 4 5 6 7

πΌππΆπΏππ·πΈ(π) 5

(24)

### Example

0 1 2 3 4 5 6 7

5 -6 3 4 -5 2 2 4

π΄

0 1 2 3 4 5 6 7

5 5

πππ(π)

0 1 2 3 4 5 6 7

5 -1

πΌππΆπΏππ·πΈ(π)

(25)

### Example

0 1 2 3 4 5 6 7

5 -6 3 4 -5 2 2 4

π΄

0 1 2 3 4 5 6 7

5 5 5 7 7 7 7 10

πππ(π)

0 1 2 3 4 5 6 7

5 -1 3 7 2 4 6 10

πΌππΆπΏππ·πΈ(π)

(26)

### Pseudocode

int maxSubarraySum(int[] A) int n=A.length

int[] OPT = new int[n]

int[] Inc = new int[n]

inc[0]=A[0]; OPT[0] = max{A[0],0}

for(int i=0;i<n;i++)

inc[i]=max{A[i], A[i]+inc[i-1]}

OPT[i]=max{inc[i], opt[i-1]}

endFor

return OPT[n-1]

(27)

### Recursive Thinking In General

As before, the hardest part is designing the recurrence.

It sometimes helps to think from multiple different angles.

Top-down: Whatβs the first step to take?

Baby Yoda will first go left or down. Use recursion to find out which of left or down is better.

The farthest right operation in the string transformation will be one of insert, delete, substitute, match for free. Use recursion to find out which is best.

(28)

### Recursive Thinking In General

Bottom-Up: What information could a recursive call give me that would help?

How does a path through most of the map help Baby Yoda?

Well we just need to know the values one left and one down.

The edit distance between which strings would help us compute the edit distance between our strings?

Well if we know the distance between π₯1 β¦ π₯πβ1 and π¦1 β¦ π¦πβ1 then that would tell us what happens if we substituteβ¦that might lead you to insertions and deletions too.

(29)

### Recursive Thinking In General

Some people refer to the βOptimal Substructure Propertyβ

From the optimum (most eggs, fewest number of string operations) for a slightly smaller problem (Baby Yoda starting closer to the end, slightly smaller strings), we need to be able to build up the optimum for the full problem.

(30)

### Longest Increasing Subsequence

Longest set of (not necessarily consecutive) elements that are increasing

5 is optimal for the array above

(indices 1,2,3,6,7; elements β6,3,6,8,10)

For simplicity β assume all array elements are distinct.

0 1 2 3 4 5 6 7

5 -6 3 6 -5 2 8 10

Updating...

## References

Updating...

Related subjects :