CSE 326: Data Structures: Sorting

(1)

CSE 326: Data Structures:

Sorting

Lecture 16: Friday, Feb 14, 2003

(2)

Review: QuickSort

procedure quickSortRecursive (Array A, int left, int right) if (left == right) return;

int pivot = choosePivot(A, left, right);

/* partition A s.t.:

A[left], A[left+1], …, A[i]  pivot A[i+1], A[i+2], …, A[right]  pivot

*/

quickSortRecursive(A, left, i);

quickSortRecursive(A, i+1, right);

}

procedure quickSortRecursive (Array A, int left, int right) if (left == right) return;

int pivot = choosePivot(A, left, right);

/* partition A s.t.:

A[left], A[left+1], …, A[i]  pivot A[i+1], A[i+2], …, A[right]  pivot

*/

quickSortRecursive(A, left, i);

quickSortRecursive(A, i+1, right);

}

(3)

Review: The Partition

i = left; j = right;

repeat { while (A[i] < pivot) i++;

while (A[j] > pivot) j--;

if (i<j) {^swap(A[i], A[j]);

i++; j++;}

else break;

}

i = left; j = right;

repeat { while (A[i] < pivot) i++;

while (A[j] > pivot) j--;

if (i<j) {^swap(A[i], A[j]);

i++; j++;}

else break;

}

There exists a sentinel A[k] pivot

A[left] … A[i-1] A[i] … A[j] … A[right]

 pivot  pivot

Why do we need i++, j+

+ ?

(4)

Review: The Partition

 pivot

A[left] … A[j] … A[i] … A[right]

 pivot

At the end:

Q: How are

these elements ? A: They are = pivot !

quickSortRecursive(A, left, j);

quickSortRecursive(A, i, right);

quickSortRecursive(A, left, j);

quickSortRecursive(A, i, right);

(5)

Why is QuickSort Faster than Merge Sort?

• Quicksort typically performs more comparisons than Mergesort, because partitions are not always perfectly balanced

– Mergesort – n log n comparisons

– Quicksort – 1.38 n log n comparisons on average

• Quicksort performs many fewer copies, because on average half of the elements are on the correct side of the partition – while Mergesort copies every element when merging

– Mergesort – 2n log n copies (using “temp array”)

n log n copies (using “alternating array”)

– Quicksort – n/2 log n copies on average

(6)

Stable Sorting Algorithms

Typical sorting scenario:

• Given N records: R[1], R[2], ..., R[N]

• They have N keys: R[1].A, ..., R[N].A

• Sort the records s.t.:R[1].A  R[2].A  ...  R[N].A

A sorting algorithm is stable if:

• If i < j and R[i].A = R[j].A

then R[i] comes before R[j] in the output

(7)

Stable Sorting Algorithms

Which of the following are stable sorting algorithms ?

• Bubble sort

• Insertion sort

• Selection sort

• Heap sort

• Merge sort

• Quick sort

(8)

8

Stable Sorting Algorithms

Which of the following are stable sorting algorithms ?

• Bubble sort yes

• Insertion sort yes

• Selection sort yes

• Heap sort no

• Merge sort no

• Quick sort no

We can always transform a non-stable sorting algorithm into a stable one

(9)

Detour: Computing the Median

• The median of A[1], A[2], …, A[N] is some A[k] s.t.:

– There exists N/2 elements  A[k]

– There exists N/2 elements  A[k]

• Think of it as the perfect pivot !

• Very important in applications:

– Median income v.s. average income – Median grade v.s. average grade

• To compute: sort A[1], …, A[N], then median=A[N/2]

– Time O(N log N)

• Can we do it in O(N) time ?

(10)

Detour: Computing the Median

int medianRecursive(Array A, int left, int right) { if (left==right) return A[left];

. . . Partition . . .

if N/2  j return medianRecursive(A, left, j);

if N/2  i return medianRecursive(A, i, right);

return pivot }

Int median(Array A, int N) { return medianRecursive(A, 0, N-1); } int medianRecursive(Array A, int left, int right)

{ if (left==right) return A[left];

. . . Partition . . .

if N/2  j return medianRecursive(A, left, j);

if N/2  i return medianRecursive(A, i, right);

return pivot }

Int median(Array A, int N) { return medianRecursive(A, 0, N-1); }

 pivot

A[left] … A[j] … A[i] … A[right]

 pivot

Why

?

(11)

Detour: Computing the Median

• Best case running time:

T(N) = T(N/2) + cN

= T(N/4) + cN(1 + 1/2)

= T(N/8) + cN(1 + 1/2 + 1/4) = . . .

= T(1) + cN (1 + 1/2 + 1/4 + … 1/2

^k

) = O(N)

• Worst case = O(N

²

)

• Average case = O(N)

• Question: how can you compute the median in O(N) worst case

time ? Note: it’s tricky.

(12)

Back to Sorting

• Naïve sorting algorithms:

– Bubble sort, insertion sort, selection sort – Time = O(N

²

)

• Clever sorting algorithms:

– Merge sort, heap sort, quick sort – Time = O(N log N)

• I want to sort in O(N) !

– Is this possible ?

(13)

Could We Do Better?

• Consider any sorting algorithm based on comparisons

• Run it on A[1], A[2], ..., A[N]

– Assume they are distinct

• At each step it compares some A[i] with some A[j]

– If A[i] < A[j] then it does something...

– If A[i] > A[j] then it does something else...

 Decision Tree !

(14)

Decision tree to sort list A,B,C

A<B

B<C

A<C C

<A C<B

B<A

A<C C

<A

B<C C

<B

A < B B < A

A < B C < B A , B , C .

A , C , B . C , A , B .

B , A , C . B < A C < A

B , C , A . C , B , A

L e g e n d

f a c t s

I n t e r n a l n o d e , w i t h f a c t s k n o w n s o f a r

A , B , C

L e a f n o d e , w i t h o r d e r i n g o f A , B , C

C<A E d g e , w i t h r e s u l t o f o n e c o m p a r i s o n

Every possible execution of the algorithm corresponds to a root-to-leaf

(15)

Max depth of the decision tree

• How many permutations are there of N numbers?

• How many leaves does the tree have?

• What’s the shallowest tree with a given number of leaves?

• What is therefore the worst running time (number of

comparisons) by the best possible sorting algorithm?

(16)

Max depth of the decision tree

• How many permutations are there of N numbers?

N!

• How many leaves does the tree have?

N!

• What’s the shallowest tree with a given number of leaves?

log(N!)

• What is therefore the worst running time (number of comparisons) by the best possible sorting algorithm?

log(N!)

(17)

Stirling’s approximation

n

e n n

n 



 



 2  

!

log( !) log 2

log( 2 ) lo g ( log )

n

n n n

e

n n n n

e



   

          

   

           

 

At least one branch in the

tree has this

(18)

If you forget Stirling’s formula...

 

1 1

1 (log !) (ln !) ( ln ) ( lo ln ! ln1 ln

g 2 ... ln

ln ln

ln ln 1

)

n n

k

n

n n

k x dx

x

n n n n n

x

n n n n



   

 

      

   

 

Theorem:

Every algorithm that sorts by comparing keys takes (n log n) time

(19)

Bucket Sort

• Now let’s sort in O(N)

• Assume:

A[0], A[1], …, A[N-1] {0, 1, …, M-1}

M = not too big

• Example: sort 1,000,000 person records on the first character of their last names:

– Hence M = 128 (in practice: M = 27)

(20)

Bucket Sort

int bucketSort(Array A, int N) { for k = 0 to M-1

Q[k] = new Queue;

for j = 0 to N-1

Q[A[j]].enqueue(A[j]);

Result = new Queue;

for k = 0 to M-1

Result = Result.append(Q[k]);

return Result;

int bucketSort(Array A, int N) { for k = 0 to M-1

Q[k] = new Queue;

for j = 0 to N-1

Q[A[j]].enqueue(A[j]);

Result = new Queue;

for k = 0 to M-1

Result = Result.append(Q[k]);

return Result;

}

Stable

sorting !

(21)

Bucket Sort

• Running time: O(M+N)

• Space: O(M+N)

• Recall that M << N, hence time = O(N)

• What about the Theorem that says sorting takes

(N log N) ?? This is not real

sorting, because

(22)

Radix Sort

• I still want to sort in time O(N): non-trivial keys

• A[0], A[1], …, A[N-1] are strings

– Very common in practice

• Each string is:

c

d-1

c

d-2

…c

1

c

0

,

where c

0

, c

1

, …, c

d-1

{0, 1, …, M-1}

M = 128

• Other example: decimal numbers

(23)

RadixSort

• Radix = “The base of a number system”

(Webster’s dictionary)

– alternate terminology: radix is number of bits needed to represent 0 to base-1; can say “base 8” or

“radix 3”

• Used in 1890 U.S.

census by Hollerith

• Idea: BucketSort on

each digit, bottom up.

(24)

The Magic of RadixSort

• Input list:

126, 328, 636, 341, 416, 131, 328

• BucketSort on lower digit:

341, 131, 126, 636, 416, 328, 328

• BucketSort result on next-higher digit:

416, 126, 328, 328, 131, 636, 341

• BucketSort that result on highest digit:

126, 131, 328, 328, 341, 416, 636

(25)

Inductive Proof that RadixSort Works

• Keys: d-digit numbers, base B

– (that wasn’t hard!)

• Claim: after i

^th

BucketSort, least significant i digits are sorted.

– Base case: i=0. 0 digits are sorted.

– Inductive step: Assume for i, prove for i+1.

Consider two numbers: X, Y. Say Xi is i^th digit of X:

• Xi+1 < Yi+1 then i+1^th BucketSort will put them in order

• Xi+1 > Yi+1 , same thing

• Xi+1 = Yi+1 , order depends on last i digits. Induction hypothesis says already sorted for these digits because BucketSort is stable

(26)

Radix Sort

int radixSort(Array A, int N) { for k = 0 to d-1

A = bucketSort(A, on position k) }

int radixSort(Array A, int N) { for k = 0 to d-1

A = bucketSort(A, on position k) }

Running time: T = O(d(M+N)) = O(dN) = O(Size)

(27)

Radix Sort

35 53 55 33 52 32 25

Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]

53 33 35 55 25

52 32 53 33 35 55 25

52 32

Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]

32 33 35

25 52 53 55

A=

(28)

Running time of Radixsort

• N items, D digit keys of max value M

• How many passes?

• How much work per pass?

• Total time?

(29)