Divide & Conquer 2
Announcements
HW2 has a separate box for every problem – it’ll take you a bit longer to upload to gradescope (but will make life easier for the TAs).
Box also available for the resubmit of an old problem – when you submit, please also fill out this form.
P1/2 Luna recorded an example of low numbers, since we got a lot of questions in OH.
P3: The “only one DFS” requirement will be considered satisfied iff your overall running time is Θ(𝑉 + 𝐸).
Deadline (for all of HW2) delayed to Thursday, since we put out so many clarifications over the last 24 hours. HW3 still out tonight, due next Wed.
Another divide and conquer
Maximum contiguous subarray sum
Given: an array of integers (positive and negative), find the indices that give the maximum contiguous subarray sum.
Find 𝑖, 𝑗 that maximize 𝐴 𝑖 + 𝐴 𝑖 + 1 + ⋯ + 𝐴[𝑗]
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Sum: 8
Another divide and conquer
Notice, it’s sometimes a good idea to include negative numbers! They might let you access larger positive numbers and give you a net
increase.
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Sum: 8
Edge Cases
We’ll allow for 𝑖 = 𝑗. In that case, 𝐴[𝑖] is the sum.
We’ll also allow for 𝑗 < 𝑖. In that case the sum is 0.
This is the best option if and only if all the entries are negative.
0 1 2 3 4 5 6 7
-3 -2 -4 -1 -3 -10 -6 -4
Sum: 0
Maximum Contiguous Subarray Sum
Brute force: How many subarrays to check? Θ(𝑛2)
How long does it take to check a subarray?
If you keep track of partial sums, the overall algorithm can take Θ(𝑛2) time.
(If you calculated from scratch every time it would take Θ(𝑛3) time) Can we do better?
Maximum Contiguous Subarray
1. Divide instance into subparts.
2. Solve the parts recursively.
3. Conquer by combining the answers
1. Split the array in half
2. Solve the parts recursively.
3. Just take the max?
Conquer
If the optimal subarray:
is only on the left – handled by the first recursive call.
is only on the right – handled by the second recursive call.
crosses the middle – TODO
Do we have to check all pairs 𝑖, 𝑗?
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Conquer
Subarrays that cross have to be handled Do we have to check all pairs 𝑖, 𝑗?
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Crossing Subarrays
Do we have to check all pairs 𝑖, 𝑗?
Sum is 𝐴 𝑛
2 − 1 + 𝐴 𝑛
2 − 2 + ⋯ + 𝐴 𝑖 + 𝐴 𝑛
2 + 𝐴 𝑛
2 + 1 + ⋯ 𝐴[𝑗]
𝑖, 𝑗 affect the sum. But they don’t affect each other.
Calculate them separately!
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Crossing Subarrays
Do we have to check all pairs 𝑖, 𝑗?
Best 𝑖? Iterate with 𝑖 decreasing, find the maximum. 𝑖 = 1 has sum 5.
Time to find 𝑖? Θ(𝑛)
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Crossing Subarrays
Do we have to check all pairs 𝑖, 𝑗?
Best 𝑗? Iterate with 𝑗 increasing, find the maximum. j = 4 has sum 3.
Time to find 𝑗? Θ(𝑛)
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Crossing Subarrays
Do we have to check all pairs 𝑖, 𝑗?
Best crossing array From the i you found to the 𝑗 you found.
Time to find? Θ(𝑛) total.
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Finishing the recursive call
Overall:
Compare sums from recursive calls and 𝐴 𝑖 + ⋯ + 𝐴[𝑗], take max.
0 1 2 3 4 5 6 7
-3 2 4 -1 3 -10 6 -4
Running Time
What does the conquer step do?
Two separate Θ(𝑛) loops, then a constant time comparison.
So
𝑇 𝑛 = ቐ2𝑇 𝑛
2 + Θ 𝑛 if 𝑛 ≥ 2
Θ 1 otherwise
Master Theorem says Θ(𝑛 log 𝑛)
MaxContigSubarraySum(int[] A, int start, int end) if(end – start < 0) //no elements
return 0
else if(end == start) //one element
return max(A[start], 0) //allow for empty int mid = start + (start – end)/2
int leftSum = MaxContigSubarraySum(A, start, mid) int rightSum = MaxContigSubarraySum(A, mid+1, end) //find best split sum
int bestSumSoFar=0; int bestISoFar = mid+1; int runningSum=0;
for(int i=mid; i >= start; i--) runningSum+=A[i];
if(runningSum > bestSumSoFar) bestISoFar=i;
bestSumSoFar=runningSum;
endIf endFor
bestSumSoFar=0; int bestJSoFar=mid; runningSum=0;
for(int j=mid+1; j <= end; j++_
runningSum+=A[j];
if(runningSum > bestSumSoFar) bestJSoFar=j
bestSumSoFar=runningSum EndIf
EndFor EndIf
splitSum = 0; for(int k = bestISoFar; k <= bestJSoFar; k++) splitSum+=A[k];
return max(leftSum, rightSum, splitSum)
One More problem
We want to calculate 𝑎𝑛%𝑏 for a very big 𝑛. What do you do?
“Naïve” algorithm” – for-loop: multiply a running product by 𝑎 and modding by 𝑏 in each iteration.
Time? 𝑂(𝑛)
“Naïve” is computer-scientist-speak for “first algorithm you’d think of.”
Thinking of it doesn’t mean you’re naïve. It means you know your fundamentals. And no one told you the secret magic speedup trick.
A better plan
What’s about half of the work?
𝑎𝑛/2%𝑏 is about half.
DivConqExponentiation(int a, int b, int n) if(n==0) return 1
if(n==1) return a%b
int k = divConqExponentiation(a,b,n/2) /*int div*/
if(n%2==1) return (k*k*a)%b return (k*k)%b
Activity
Write a recurrence to describe the running time of this code. What’s the big-O?
DivConqExponentiation(int a, int b, int n) if(n==0) return 1
if(n==1) return a%b
int k = divConqExponentiation(a,b,n/2) /*int div*/
if(n%2==1) return (k*k*a)%b return (k*k)%b
Go to pollev.com/Robbie so I know how long to explain
Master Theorem
𝑇 𝑛 = ቐ
𝑑 if 𝑛 is at most some constant 𝑎𝑇 𝑛
𝑏 + 𝑓 𝑛 otherwise
Given a recurrence of the following form, where 𝑎, 𝑏, 𝑐, and 𝑑 are constants:
Where 𝑓 𝑛 is Θ 𝑛𝑐
𝑇 𝑛 ∈ Θ 𝑛𝑐 log𝑏 𝑎 < 𝑐
log𝑏 𝑎 = 𝑐 𝑇 𝑛 ∈ Θ 𝑛𝑐 log 𝑛 log𝑏 𝑎 > 𝑐 𝑇 𝑛 ∈ Θ 𝑛log𝑏 𝑎 If
If If
then then then
Running Time?
𝑇 𝑛 = ቐ 𝑇 𝑛
2 + Θ 1 if 𝑛 ≥ 2
Θ 1 otherwise
Which is Θ(log 𝑛) time.
Much faster than 𝑂(𝑛).
Fun Fact: RSA encryption (one of the schemes most used for internet security) needs exactly this sort of “raise to a large power, mod b”
operation.
When to use Divide & Conquer
Your problem has a structure that makes it easy to split up
Usually in half, but not always…
You can split “what you’re looking for” (subarrays, inversions, etc.) into
“just in one of the parts” and “split across parts”
The “split” ones can be studied faster than brute force.
Classic Divide & Conquer
Classic Applications
There are a few really famous divide & conquer algorithms where the way to recurse is very clever.
The point of the next few examples is not to teach you really useful tricks (they often don’t generalize to new problems).
It’s partially to show you that sometimes being really clever gives you a really big improvement
And partially because these are “standard” in algorithms classes, so you should at least have heard of these algorithms.
Classic Application
Suppose you need to multiply really big numbers.
Much bigger than ints
Split the n bit numbers in half
Think of them as written in base 2𝑛/2
What would the “normal” multiplication algorithm do?
x_1 x_0
y_1 y_0
x_0y_0 x_1y_0
x_0y_1 x_1y_1
4 multiplications, i.e. 4 recursive calls.
Classic Application
If n bits is too many to multiply in one step (e.g. it’s more than one byte, or whatever your processor does in one cycle)
Recurse! Running time?
𝑇 𝑛 = ቐ 4𝑇 𝑛
2 + 𝑂 𝑛 if 𝑛 is large 𝑂 1 if 𝑛 fits in one byte
x_1 x_0
y_1 y_0
x_0y_0 x_1y_0
x_0y_1 x_1y_1
Why 𝑂 𝑛 ? It takes 𝑂(𝑛) time to add up 𝑂(𝑛) bit numbers – they have 𝑂(𝑛) bytes!
(Why have you never seen this before? We assumed our numbers were ints, where the number of bytes is a constant)
Overall running time is Θ 𝑛2
Clever Trick
We need to find x_1y_0 + x_0y_1.
Does that look familiar? It’s the middle to terms when you FOIL Define 𝛼 = 𝑥0 + 𝑥1 , 𝛽 = (𝑦0 + 𝑦1)
𝛼 ⋅ 𝛽 = 𝑥0𝑦0 + 𝑥1𝑦0 + 𝑥0𝑦1 + 𝑥1𝑦1 So 𝛼𝛽 − 𝑥0𝑦0 − 𝑥1𝑦1 = 𝑥1𝑦0 + 𝑥0𝑦1
What do we need to find the overall multiplication?
𝑥0𝑦0 + (𝛼𝛽 − 𝑥0𝑦0 − 𝑥1𝑦1) ⋅ 2
𝑛
2 + 𝑥1𝑦1 ⋅ 2𝑛
𝑥0𝑦0, 𝑥1𝑦1 and 𝛼𝛽 are enough to calculate the overall answer! Only 3 multiplies of n/2 bits!
Running Time
𝑇 𝑛 = ቐ 3𝑇 𝑛
2 + 𝑂 𝑛 if 𝑛 is large 𝑂 1 if 𝑛 fits in one byte
log2 3 > 1,
So running time is 𝑂 𝑛log2 3 Or about 𝑂 𝑛1.585
Strassen’s Algorithm
Apply that “save a multiplication” idea to multiplying matrices, and you can also get a speedup.
Called Strassen’s Algorithm
Instead of 𝑂 𝑛3 time, can get down to 𝑂 𝑛log2 7 ≈ 𝑂 𝑛2.81 .
Lots of more clever ideas have gotten matrix multiplication even faster theoretically. In practice, the constant factors are too high.