AlgorithmStrategies

(1)

Algorithm Strategies

(2)

Greedy Algorithms

• _{Make best choice based on some short term}

criteria

• Choice is optimal at each step of the algorithm

• _{Final solution may not be optimal}

– _{Choosing a non-optimal solution to start may lead}

(3)

Greedy Algorithms

• _{Must have}

– _{Greedy-choice property: a globally optimal solution}

can be arrived at by making a locally optimal choice

– _{Optimal substructure: an optimal solution to a}

(4)

Greedy Algorithm - Examples

• _{Restaurants using plastic cups to save money}

– _{Will initially save money} – _{Will turn away customers}

– _{Will lose money overall when customers are lost}

• Protecting a piece in chess that will be lost no matter what

– Keep sacrificing pieces to save the queen

– _{Will lose queen anyway}

(5)

Greedy Algorithm Properties

• _{Fast and simple} • Used in heuristics

– _{Heuristic vs. Algorithm}

• May not give overall optimal solution

– _{May find local maximum/minimum but not global}

(6)

Local vs. Global (minimum)

• _{Rolling a ball down al hill may arrive at a local}

minimum

(7)

Greedy Algorithms

• Task scheduling

• Activity Selection

• Various AI algorithms

• _{Various Graph Theory Algorithms}

• Heuristics that pick the best at the time

(8)

Task Scheduling

• Given jobs j₁, j₂, j₃, ..., j_n with known running times t₁, t₂, t₃, ..., t_n – what is the best way to schedule the jobs to minimize average completion time?

Job Time

j1 15

j2 8

j3 3

(9)

Scheduling

Average completion time = (15+23+26+36)/4 = 25

Average completion time = (3+11+21+36)/4 = 17.75

Job Name J1 J2 J3 J4

Completion

Time 15 23 26 36

Job Name J3 J2 J4 J1

Completion

(10)

Scheduling

• Greedy-choice property:

– If the shortest job (j₃) does not go first

• All the other jobs before the first job will complete 3 time units faster

• j₃ will be postponed by time it took to complete all the other jobs that ran before j₃

• Optimal substructure:

– If the shortest job is removed from optimal solution

• The remaining solution for the other jobs is optimal

(11)

Activity Selection Problem

• _{Given a time frame and a set of mutually}

exclusive activities and their start and end

times, what is the largest number of activities that can be run in the time frame?

(12)

Activity Selection Problem

• _{Sort the activities by earliest finish time}

• Choose the activities by earliest finish time

• Keep choosing non-overlapping activities

• Stop when the time frame is reached

(13)

Example

(14)

Approximate Bin Packing

• N items of sizes s₁, s₂, ..., s_N

• 0 < s_i<= 1

• _{Goal: pack into fewest number of bins of size 1} • _{NP-complete problem, but we can use greedy}

algorithms to produce near optimal solutions

• _{Knapsack problem}

– Items have value

(15)

Example – Optimal Packing

0.8

0.2

0.3

0.7

0.5

0.4 0.1

•_{Input: 0.2, 0.5, 0.4, 0.7, 0.1, 0.3, 0.8}

(16)

Online vs Offline Algorithms

• _Online

– _{Process one item at a time} – _{Process data as it comes in}

– _{Must make a decision immediately}

• Off-line

(17)

Online Bin-Packing

• _{Look at each item one at a time in order}

• Decide where to put that item before looking at the next item

(18)

Online Bin-Packing

• Online algorithms cannot guarantee optimal solution

– Problem: cannot know when input will end

– M small items ½-ε – M large items ½+ε

– Can fit into M bins with 1 large and 1 small in each bin

– If all small items come first, place in M separate bins

– If input is only M small items, we have used twice as many bins as necessary

– There are inputs that force any online bin-packing

(19)

Online Bin Packing Algorithms

• _{Next fit} • First fit

(20)

Next Fit

• Algorithm

– If the item fits in the current bin, put it there

– Otherwise place it in a new bin

– Never look back at previous bins • Running time = O(n)

• Let M be the optimal number of bins required to pack a list of items. Then next fit never

uses more than 2M bins.

(21)

Next Fit Example

(22)

First Fit

• Algorithm

– Scan all bins and place the item in first bin large enough to hold it

– If no bin is large enough, create a new bin

• Running time = O(nlog(n))

– n = number of items

– log(n) = upper bound on search time

• Assumes you store the bins in sorted order based on capacity • Let M be the optimal number of bins required to

(23)

First Fit Example

(24)

Best Fit

• _Algorithm

– _{Scan all bins and place the item in the bin with}

tightest fit

– _{If no bin is large enough, create a new bin}

(25)

Best Fit Example

(26)

Offline Bin Packing

• _{Sort items in decreasing order for easier}

placement of large items

• Apply first fit or best fit algorithm

• _{Let M be the optimal number of bins required}

(27)

(28)

Divide and Conquer

• _Divide

– _{Divide the problem into smaller sub-problems}

• _Conquer

– _{Solve the sub-problems}

– _{Can be solved by breaking them up into more}

sub-problems to solve • _Combine

(29)

Divide And Conquer

• _{What are some divide and conquer algorithms}

(30)

Divide And Conquer

• _Covered

– _Quicksort – _{Merge Sort} – _{Binary Search}

• Others

– _{Matrix Multiplication}

(31)

Quicksort

• _{What are the steps?} • Divide

– _?

• Conquer

– _?

• Combine

(32)

Quicksort

• _{What are the steps?}

• Divide

– Partition into two sub groups

– _{One greater than the pivot}

– One less than or equal to the pivot • Conquer

– _{Recursively sort the sub-lists}

• Combine

(33)

Merge Sort

– _?

• Conquer

– _?

• Combine

(34)

Merge Sort

– _{Recursively split each list into two equal-sized}

sub-lists • Conquer

– _{Sort the sub-lists}

• Combine

(35)

Binary Search

– _?

• Conquer

– _?

• Combine

(36)

Binary Search

– _{Check the middle element}

• Conquer

– _{Search one sub-list}

• Combine