1
CSE 326: Data Structures:
Sorting
Lecture 16: Friday, Feb 14, 2003
2
Review: QuickSort
procedure quickSortRecursive (Array A, int left, int right)
if (left == right) return;
int pivot = choosePivot(A, left, right);
/* partition A s.t.:
A[left], A[left+1], …, A[i] ≤ pivot A[i+1], A[i+2], …, A[right] ≥ pivot
*/
quickSortRecursive(A, left, i);
quickSortRecursive(A, i+1, right);
}
3
Review: The Partition
i = left; j = right;
repeat { while (A[i] < pivot) i++;
while (A[j] > pivot) j--;
if (i<j) {swap(A[i], A[j]);
i++; j++;}
else break;
}
There exists a sentinel A[k]≥ pivot
A[left] … A[i-1] A[i] … A[j] … A[right]
≤ pivot ≥ pivot
Why do we need i++,
j++ ?
4
Review: The Partition
≥ pivot
A[left] … A[j] … A[i] … A[right]
≤ pivot
At the end:
Q: How are
these elements ? A: They are = pivot
!
quickSortRecursive(A, left, j);
quickSortRecursive(A, i, right);
5
Why is QuickSort Faster than Merge Sort?
• Quicksort typically performs more comparisons than Mergesort, because partitions are not always perfectly balanced
– Mergesort – n log n comparisons
– Quicksort – 1.38 n log n comparisons on average
• Quicksort performs many fewer copies, because on average half of the elements are on the correct side of the partition – while Mergesort copies every element when merging
– Mergesort – 2n log n copies (using “temp array”)
n log n copies (using “alternating array”)
– Quicksort – n/2 log n copies on average
6
Stable Sorting Algorithms
Typical sorting scenario:
• Given N records: R[1], R[2], ..., R[N]
• They have N keys: R[1].A, ..., R[N].A
• Sort the records s.t.:R[1].A ≤ R[2].A ≤ ... ≤ R[N].A A sorting algorithm is stable if:
• If i < j and R[i].A = R[j].A
then R[i] comes before R[j] in the output
7
Stable Sorting Algorithms
Which of the following are stable sorting algorithms
?
• Bubble sort
• Insertion sort
• Selection sort
• Heap sort
• Merge sort
• Quick sort
8
Stable Sorting Algorithms
Which of the following are stable sorting algorithms
?
• Bubble sort yes
• Insertion sort yes
• Selection sort yes
• Heap sort no
• Merge sort no
• Quick sort no
We can always transform a non-stable sorting algorithm into a stable one
How ?
9
Detour: Computing the Median
• The median of A[1], A[2], …, A[N] is some A[k] s.t.:
– There exists N/2 elements ≤ A[k]
– There exists N/2 elements ≥ A[k]
• Think of it as the perfect pivot !
• Very important in applications:
– Median income v.s. average income – Median grade v.s. average grade
• To compute: sort A[1], …, A[N], then median=A[N/2]
– Time O(N log N)
• Can we do it in O(N) time ?
10
Detour: Computing the Median
int medianRecursive(Array A, int left, int right) { if (left==right) return A[left];
. . . Partition . . .
if N/2 ≤ j return medianRecursive(A, left, j);
if N/2 ≥ i return medianRecursive(A, i, right);
return pivot }
Int median(Array A, int N) { return medianRecursive(A, 0, N-1); }
≥ pivot
A[left] … A[j] … A[i] … A[right]
≤ pivot
Why
?
11
Detour: Computing the Median
• Best case running time:
T(N) = T(N/2) + cN
= T(N/4) + cN(1 + 1/2)
= T(N/8) + cN(1 + 1/2 + 1/4) = . . .
= T(1) + cN (1 + 1/2 + 1/4 + … 1/2
k) = O(N)
• Worst case = O(N
2)
• Average case = O(N)
• Question: how can you compute the median in
O(N) worst case time ? Note: it’s tricky.
12
Back to Sorting
• Naïve sorting algorithms:
– Bubble sort, insertion sort, selection sort – Time = O(N
2)
• Clever sorting algorithms:
– Merge sort, heap sort, quick sort – Time = O(N log N)
• I want to sort in O(N) !
– Is this possible ?
13
Could We Do Better?
• Consider any sorting algorithm based on comparisons
• Run it on A[1], A[2], ..., A[N]
– Assume they are distinct
• At each step it compares some A[i] with some A[j]
– If A[i] < A[j] then it does something...
– If A[i] > A[j] then it does something else...
Decision Tree !
14
Decision tree to sort list A,B,C
Every possible execution of the algorithm corresponds to a root-to-leaf
path in the tree.
15
Max depth of the decision tree
• How many permutations are there of N numbers?
• How many leaves does the tree have?
• What’s the shallowest tree with a given number of leaves?
• What is therefore the worst running time (number of
comparisons) by the best possible sorting algorithm?
16
Max depth of the decision tree
• How many permutations are there of N numbers?
N!
• How many leaves does the tree have?
N!
• What’s the shallowest tree with a given number of leaves?
log(N!)
• What is therefore the worst running time (number of comparisons) by the best possible sorting algorithm?
log(N!)
17
Stirling’s approximation
At least one branch in
the
tree has this
depth
18
If you forget Stirling’s formula...
Theorem:
Every algorithm that sorts by comparing keys takes Ω(n log n) time
19
Bucket Sort
• Now let’s sort in O(N)
• Assume:
A[0], A[1], …, A[N-1] ∈{0, 1, …, M-1}
M = not too big
• Example: sort 1,000,000 person records on the first character of their last names:
– Hence M = 128 (in practice: M = 27)
20
Bucket Sort
int bucketSort(Array A, int N) { for k = 0 to M-1
Q[k] = new Queue;
for j = 0 to N-1
Q[A[j]].enqueue(A[j]);
Result = new Queue;
for k = 0 to M-1
Result = Result.append(Q[k]);
return Result;
}
Stable
sorting !
21
Bucket Sort
• Running time: O(M+N)
• Space: O(M+N)
• Recall that M << N, hence time = O(N)
• What about the Theorem that says sorting takes Ω (N log N) ?? This is not real
sorting, because it’s for trivial
keys
22
Radix Sort
• I still want to sort in time O(N): non-trivial keys
• A[0], A[1], …, A[N-1] are strings
– Very common in practice
• Each string is:
c
d-1c
d-2…c
1c
0,
where c
0, c
1, …, c
d-1∈{0, 1, …, M-1}
M = 128
• Other example: decimal numbers
23
RadixSort
• Radix = “The base of a number system”
(Webster’s dictionary)
– alternate terminology: radix is number of bits needed to represent 0 to base-1; can say “base 8” or
“radix 3”
• Used in 1890 U.S.
census by Hollerith
• Idea: BucketSort on
each digit, bottom up.
24
The Magic of RadixSort
• Input list:
126, 328, 636, 341, 416, 131, 328
• BucketSort on lower digit:
341, 131, 126, 636, 416, 328, 328
• BucketSort result on next-higher digit:
416, 126, 328, 328, 131, 636, 341
• BucketSort that result on highest digit:
126, 131, 328, 328, 341, 416, 636
25
Inductive Proof that RadixSort Works
• Keys: d-digit numbers, base B
– (that wasn’t hard!)
• Claim: after i
thBucketSort, least significant i digits are sorted.
– Base case: i=0. 0 digits are sorted.
– Inductive step: Assume for i, prove for i+1.
Consider two numbers: X, Y. Say X
iis i
thdigit of X:
• X
i+1< Y
i+1then i+1
thBucketSort will put them in order
• X
i+1> Y
i+1, same thing
• X
i+1= Y
i+1, order depends on last i digits. Induction
hypothesis says already sorted for these digits because
BucketSort is stable
26
Radix Sort
int radixSort(Array A, int N) { for k = 0 to d-1
A = bucketSort(A, on position k) }
Running time: T = O(d(M+N)) = O(dN) = O(Size)
27
Radix Sort
35 53 55 33 52 32 25
Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]
53 33 35 55 25
52 32 53 33 35 55 25
52 32
Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]
32 33 35
25 52 53 55
25 32 33 35 52 53 55
A=
A=
A=
28
Running time of Radixsort
• N items, D digit keys of max value M
• How many passes?
• How much work per pass?
• Total time?
29
Running time of Radixsort
• N items, D digit keys of max value M
• How many passes? D
• How much work per pass? N + M
– just in case M>N, need to account for time to empty out buckets between passes
• Total time? O( D(N+M) )
30
Radix Sort
• What is the size of the input ? Size = DN
• Radix sort takes time O(Size) !!
cD-1 cD-2 … c0
A[0] ‘S’ ‘m’ ‘i’ ‘t’ ‘h’
A[1] ‘J’ ‘o’ ‘n’ ‘e’ ‘s’
… A[N-1]
31
Radix Sort
• Variable length strings:
• Can adapt Radix Sort to sort in time O(Size) !
– What about our Theorem ??
A[0]
A[1]
A[2]
A[3]
A[4]
