## Quicksort I

■

### To sort

### a[left...right]

### :

1. if left < right:

1.1. Partition a[left...right] such that:

all a[left...p-1] are less than a[p], and all a[p+1...right] are >= a[p]

## Partitioning (Quicksort II)

■

### A key step in the Quicksort algorithm is

### partitioning

### the

### array

■ We choose some (any) number p in the array to use as a pivot ■ We partition the array into three parts:

p

*numbers *
*less than *p

*numbers greater than *
*or equal to *p

## Partitioning II

■

### Choose an array value (say, the first) to use as the

### pivot

■

### Starting from the left end, find the first element

### that is greater than or equal to the pivot

■

### Searching backward from the right end, find the

### first element that is less than the pivot

■

### Interchange (swap) these two elements

■

### Repeat, searching from where we left off, until

## Partitioning

■

### To partition a[left...right]:

1. Set pivot = a[left], l = left + 1, r = right;

2. while l < r, do

2.1. while l < right & a[l] < pivot , set l = l + 1

2.2. while r > left & a[r] >= pivot , set r = r - 1

2.3. if l < r, swap a[l] and a[r]

3. Set a[left] = a[r], a[r] = pivot

## Example of partitioning

■ choose pivot:

**4**

** 3 6 9 2 4 3 1 2 1 8 9 3 5 6**

■ search:

**4**

** 3 6 9 2 4 3 1 2 1 8 9 **

**3**

**5 6**

■ swap:

**4**

** 3 **

**3**

** 9 2 4 3 1 2 1 8 9 6**

**5 6**

■ search:

**4**

** 3 3 9 2 4 3 1 2 **

**1**

** 8 9 6 5 6**

■ swap:

**4**

** 3**

**3 **

**1**

** 2 4 3 1 2 9 8 9 6 5 6**

■ search:

**4**

** 3**

**3 1 2 4 3 1 **

**2**

** 9 8 9 6 5 6**

■ swap:

**4**

** 3**

**3 1 2 **

**2**

** 3 1 4 9 8 9 6 5 6**

## The partition method (Java)

static int partition(int[] a, int left, int right) { int p = a[left], l = left + 1, r = right;

while (l < r) {

while (l < right && a[l] < p) l++; while (r > left && a[r] >= p) r--; if (l < r) {

int temp = a[l]; a[l] = a[r]; a[r] = temp; }

}

a[left] = a[r]; a[r] = p;

## The quicksort method (in Java)

static void quicksort(int[] array, int left, int right) { if (left < right) {

int p = partition(array, left, right); quicksort(array, left, p - 1);

quicksort(array, p + 1, right); }

### Analysis of quicksort—best case

■

### Suppose each partition operation divides the array

### almost exactly in half

■

### Then the depth of the recursion in

### log

2

### n

## Best case II

■

### We cut the array size in half each time

■

### So the depth of the recursion in

### log

2

### n

■

### At each level of the recursion, all the partitions at that

### level do work that is linear in

### n

■

### O(log

2

### n) * O(n) = O(n log

2### n)

■

### Hence in the average case, quicksort has time

### complexity

### O(n log

_{2}

### n)

## Worst case

■

### In the worst case, partitioning always divides the size

### n

### array into these three parts:

■ A length one part, containing the pivot itself

■ A length zero part, and

■ A length n-1 part, containing everything else

■

### We don’t recur on the zero-length part

■

### Recurring on the length

### n-1

### part requires (in the worst

## Worst case for quicksort

■

### In the worst case, recursion may be

### n

### levels deep (for

### an array of size

### n

### )

■

### But the partitioning work done at each level is still

### n

■

### O(n) * O(n) = O(n

2### )

■

### So worst case for Quicksort is

### O(n

2### )

■

### When does this happen?

■ There are many arrangements that could make this happen

■ Here are two common cases:

■ When the array is already sorted

## Typical case for quicksort

■

### If the array is sorted to begin with, Quicksort is

### terrible:

### O(n

2### )

■

### It is possible to construct other bad cases

■

### However, Quicksort is

*usually*

### O(n log

2

### n)

■

### The constants are so good that Quicksort is

### generally the fastest algorithm known

## Tweaking Quicksort

■

### Almost anything you can try to “improve”

### Quicksort will actually slow it down

■

### One

*good*

### tweak is to switch to a different

### sorting method when the subarrays get small

### (say, 10 or 12)

■ Quicksort has too much overhead for small array

sizes

■

### For large arrays, it

*might*

### be a good idea to check

### beforehand if the array is already sorted

## Picking a better pivot

■

### Before, we picked the

*first*

### element of the subarray

### to use as a pivot

■ If the array is already sorted, this results in O(n2)

behavior

■ It’s no better if we pick the last element

■

### We could do an

*optimal*

### quicksort (guaranteed

### O(n log n)

### ) if we always picked a pivot value

### that exactly cuts the array in half

■ Such a value is called a median: half of the values in the

array are larger, half are smaller

■ The easiest way to find the median is to sort the array

## Median of three

■

### Obviously, it doesn’t make sense to sort the array

### in order to find the median to use as a pivot

■

### Instead, compare just

*three*

### elements of our

### (sub)array—the first, the last, and the middle

■ Take the median (middle value) of these three as pivot ■ It’s possible (but not easy) to construct cases which will

make this technique O(n2)

■

### Suppose we rearrange (sort) these three numbers

### so that the smallest is in the first position, the

### largest in the last position, and the other in the

### middle

## Final comments

■

### Quicksort is the fastest known sorting algorithm

■

### For optimum efficiency, the pivot must be chosen

### carefully

■

### “Median of three” is a good technique for choosing the

### pivot

■

### However, no matter what you do, there will be some

## The Hiring Problem

■ You are using an employment agency to hire a new assistant. ■ The agency sends you one candidate each day.

■ You interview the candidate and must immediately decide

whether or not to hire that person. But if you hire, you must also fire your current office assistant—even if it’s someone you have recently hired.

■ Cost to interview is *c*

*i* per candidate.

■ Cost to hire is *c*

*h* per candidate.

■ You want to have, at all times, the best candidate seen so far. ■ When you interview a candidate who is better than your current

assistant, you fire the current assistant and hire the candidate.

## Pseudo-code to Model the Scenario

*Hire-Assistant (n)*

*best *← 0 ;;Candidate 0 is a least qualified sentinel candidate

* for i *← 1

**to n** **do** interview candidate *i*

* if *candidate

*i*is better than candidate

*best*

**then** *best* ← *i*

* *hire candidate

*i*

Cost Model: Slightly different from the model considered so far. However, analytical techniques are the same.

• Want to determine the total cost of hiring the best candidate.
• If *n* candidates interviewed and *m* hired, then cost is *nc _{i}+mc_{h}*.
• Have to pay nc

*to interview, no matter how many we hire. • So, focus on analyzing the hiring cost mc*

_{i}*.*

_{h}## Worst-case Analysis

■

### In the worst case, we

### hire all

*n*

### candidates

### .

■

### This happens if each candidate is better than all those

### who came before. Candidates come in increasing order

### of quality.

■

### Cost is

*nc*

*i*

### +

*nc*

*h*

### .

■

### If this happens, we fire the agency. What should happen

## Probabilistic Analysis

■

### We need a probability distribution of inputs to determine

### average-case behavior over all possible inputs.

■

### For the hiring problem, we can assume that candidates

### come in random order.

■ Assign a rank rank(i), a unique integer in the range 1 to n to

each candidate.

■ The ordered list 〈rank(1), rank(2), …, rank(n)〉 is a

permutation of the candidate numbers 〈1, 2, …, n〉.

■ Let’s assume that the list of ranks is equally likely to be any

one of the n! permutations.

■ The ranks form a uniform random permutation.

■ Determine the number of candidates hired on an average,

## Randomized Algorithm

■

### Impose a distribution

### on the inputs by using

### randomization within the algorithm.

■

### Used when

### input distribution is not known

### , or cannot be

### modeled computationally.

■

### For the hiring problem:

■ We are unsure if the candidates are coming in a random order. ■ To make sure that we see the candidates in a random order, we

make the following change.

■ The agency sends us a list of *n* candidates in advance.
■ Each day, we randomly choose a candidate to interview.

■ Thus, instead of relying on the candidates being presented in a

## Discrete Random Variables

■

### A

### random variable

### X is a

### function

### from a sample space S

### to the real numbers.

■

### If the space is finite or countably infinite, a random

### variable X is called a

### discrete random variable

### .

■

### Maps each possible outcome of an experiment to a real

### number.

■

### For a random variable

*X*

### and a real number

*x*

### , the event

*X=x *

### is {

*s*

∈*S*

: *X*(

*s*)=

*x*

### }

.■

### Pr{

*X=x*

### } = ∑

{*s*∈

*S:X*{

*s*}=x}

### Pr{

*s*

### }

*■*

*f*

### (

*x*

### ) = Pr{

*X*

### =

*x*

### } is the

### probability density function

### of the

## Discrete Random Variables

■

### Example:

■ Rolling 2 dice.

*■* *X: Sum of the values on the two dice.*

■ Pr{X=7} = Pr{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)}

## Expectation

■

### Average or mean

■

### The expected value of a discrete random variable

*X*

### is

*E*

### [

*X*

### ] = ∑

_{x }*x *

### Pr{

*X*

### =

*x*

### }

■

### Linearity of Expectation

*■* *E[X+Y] = E[X]+E[Y], for all X, Y*

*■* *E[aX+Y]=aE[X]+E[Y], for constant a and all X, Y*

■

### For mutually independent random variables

*X*

1

### ,

*X*

2### , …,

*X*

_{n}

*■* *E*[*X*

## Expectation – Example

■

### Let

*X*

### be the RV denoting the value obtained when a fair

### die is thrown. What will be the mean of

*X*

### , when the die

### is thrown

*n*

### times.

■ Let X

1, X2, …, Xn denote the values obtained during the n throws.

■ The mean of the values is (*X*

1+*X*2+…+*X*n)/*n*.

■ Since the probability of getting values 1 thru 6 is (1/6), on an

average we can expect each of the 6 values to show up (1/6)*n*

times.

■ So, the numerator in the expression for mean can be written as

(1/6)*n·*1+(1/6)*n·*2+…+(1/6)*n·*6

## Indicator Random Variables

■

### A simple yet powerful technique for computing the

### expected value of a random variable.

■

### Convenient method for converting between

### probabilities and expectations.

■

### Helpful in situations in which there may be

### dependence.

■

### Takes only 2 values, 1 and 0.

■

### Indicator Random Variable for

### an

### event A

### of a

## Indicator Random Variable

**Lemma 5.1**

### Given a sample space

*S*

### and an event

*A*

### in the sample

### space

*S*

### , let

*X*

_{A}### = I{

*A*

### }

### . Then

### E[

*X*

_{A}### ] = Pr{A}

### .

**Proof:**

Let Ā = S – A (Complement of A) Then,

E[X* _{A}*] = E[I{A}]

## Indicator RV – Example

### Problem:

### Determine the expected number of heads in

*n*

### coin flips.

### Method 1:

### Without indicator random variables.

### Let

*X*

### be the random variable for the number of heads in

*n*

### flips.

### Then, E[

*X*

### ] = ∑

_{k=0..n}

*k*

### ·Pr{

*X*

### =

*k*

### }

## Indicator RV – Example

**■**

**Method 2 :**

### Use Indicator Random Variables

■

### Define

*n*

### indicator random variables,

*X*

i

### , 1 ≤

*i*

### ≤

*n*

### .

■

### Let

*X*

i

### be the indicator random variable for the event

### that the

*i*

th### flip results in a Head.

*■*

*X*

i

### = I{the

*i*

th

_{ flip results in }

_{H}

_{H}

_{}}

■

### Then

*X*

### =

*X*

1

### +

*X*

2### + …+

*X*

*n*

### = ∑

i=1..*n*

*X*

_{i}

### .

■

### By Lemma 5.1, E[

*X*

i

### ] = Pr{H} = ½, 1 ≤

*i*

### ≤

*n*

### .

■

### Expected number of heads is E[

*X*

### ] = E[∑

_{i}_{=1..}

_{n}*X*

i

### ].

■

### By linearity of expectation, E[∑

_{i}_{=1..}

_{n}*X*

i

### ] = ∑

*i*=1..

*n*

### E[

*X*

_{i}

### ].

■

### E[

*X*

### ] = ∑

_{i}_{=1..}

_{n}### E[

*X*

## Randomized Hire-Assistant

*Randomized-Hire-Assistant (n)*

Randomly permute the list of candidates

*best *← 0 ;;Candidate 0 is a least qualified dummy candidate

* for i *← 1

**to n** **do** interview candidate *i*

* if *candidate

*i*is better than candidate

*best*

**then** *best* ← *i*

* *hire candidate

*i*

## Analysis of the Hiring Problem

(Probabilistic analysis of the deterministic algorithm)

*■*

*X*

### – RV that denotes the

### number of times we hire a new

### office assistant

### .

■

### Define indicator RV’s

*X*

1

### ,

*X*

2### , …,

*X*

n### .

*■*

*X*

i

### = I{candidate

*i *

### is hired}

### .

■

### As in the previous example,

*■* *X* = *X*

1 + *X*2 + …+ *Xn*

■ Need to compute Pr{candidate i is hired}.

■

### Pr{candidate

*i*

### is hired}

*■* *i is hired only if i is better than 1, 2,…,i-1.*

■ By assumption, candidates arrive in random order

■ Candidates 1, 2, …, *i* arrive in random order.

■ Each of the *i* candidates has an equal chance of being the best so far.
■ Pr{candidate *i* is the best so far} = 1/*i*.

■ E[*X*

## Analysis of the Hiring Problem

■ Compute E[*X*], the number of candidates we expect to

hire.

By Equation (A.7) of the sum of a harmonic series.

## Analysis of the randomized hiring

## problem

■

### Permutation of the input array results in a situation that

### is identical to that of the deterministic version.

■

### Hence, the same analysis applies.

■

### Expected hiring cost

### is hence

*O*

### (

*c*

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### What is best case running time?

■ Recursion:

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### What is best case running time?

■ Recursion:

1. Partition splits array in two sub-arrays of size n/2 2. Quicksort each sub-array

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### What is best case running time?

■ Recursion:

1. Partition splits array in two sub-arrays of size n/2 2. Quicksort each sub-array

■ Depth of recursion tree? O(log

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### What is best case running time?

■ Recursion:

1. Partition splits array in two sub-arrays of size n/2 2. Quicksort each sub-array

■ Depth of recursion tree? O(log

2n)

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### What is best case running time?

■ Recursion:

1. Partition splits array in two sub-arrays of size n/2 2. Quicksort each sub-array

■ Depth of recursion tree? O(log

2n)

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

2

### n)

## Quicksort: Worst Case

■

### Assume first element is chosen as pivot.

■

### Assume we get array that is already in order:

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot] ++too_big_index

2. While data[too_small_index] > data[pivot] --too_small_index

3. If too_big_index < too_small_index

swap data[too_big_index] and data[too_small_index] 4. While too_small_index > too_big_index, go to 1.

5. Swap data[too_small_index] and data[pivot_index]

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot] ++too_big_index

2. While data[too_small_index] > data[pivot] --too_small_index

3. If too_big_index < too_small_index

swap data[too_big_index] and data[too_small_index] 4. While too_small_index > too_big_index, go to 1.

5. Swap data[too_small_index] and data[pivot_index]

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot] ++too_big_index

2. While data[too_small_index] > data[pivot] --too_small_index

3. If too_big_index < too_small_index

swap data[too_big_index] and data[too_small_index] 4. While too_small_index > too_big_index, go to 1.

5. Swap data[too_small_index] and data[pivot_index]

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot] ++too_big_index

2. While data[too_small_index] > data[pivot] --too_small_index

3. If too_big_index < too_small_index

5. Swap data[too_small_index] and data[pivot_index]

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot] ++too_big_index

2. While data[too_small_index] > data[pivot] --too_small_index

3. If too_big_index < too_small_index

5. Swap data[too_small_index] and data[pivot_index]

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot] ++too_big_index

2. While data[too_small_index] > data[pivot] --too_small_index

3. If too_big_index < too_small_index

5. Swap data[too_small_index] and data[pivot_index]

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot] ++too_big_index

2. While data[too_small_index] > data[pivot] --too_small_index

3. If too_big_index < too_small_index

5. Swap data[too_small_index] and data[pivot_index]

2 4 10 12 13 50 57 63 10 0

pivot_index = 0

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

2

### n)

■

### Worst case running time?

■ Recursion:

1. Partition splits array in two sub-arrays:

• one sub-array of size 0

• the other sub-array of size n-1

2. Quicksort each sub-array

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

2

### n)

■

### Worst case running time?

■ Recursion:

1. Partition splits array in two sub-arrays:

• one sub-array of size 0

• the other sub-array of size n-1

2. Quicksort each sub-array

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

2

### n)

■

### Worst case running time?

■ Recursion:

1. Partition splits array in two sub-arrays:

• one sub-array of size 0

• the other sub-array of size n-1

2. Quicksort each sub-array

■ Depth of recursion tree? O(n)

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

2

### n)

■

### Worst case running time?

■ Recursion:

1. Partition splits array in two sub-arrays:

• one sub-array of size 0

• the other sub-array of size n-1

2. Quicksort each sub-array

■ Depth of recursion tree? O(n)

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

2

### n)

## Quicksort Analysis

■

### Assume that keys are random, uniformly distributed.

■

### Best case running time: O(n log

2

### n)

■

### Worst case running time: O(n

2### )!!!

## Improved Pivot Selection

### Pick median value of three elements from data array:

### data[0], data[n/2], and data[n-1].

## Improving Performance of Quicksort

■

### Improved selection of pivot.

■

### For sub-arrays of size 3 or less, apply brute force

### search:

■ Sub-array of size 1: trivial ■ Sub-array of size 2:

■ if(data[first] > data[second]) swap them