DATA STRUCTURES
AND ALGORITHMS
◻
Sorting
◻
Bubble Sort
◻
Selection Sort
◻
Insertion Sort
◻
Merge Sort
◻
Quick Sort
Sorting: Definition
Sorting: an operation that segregates items into groups
according to specified criterion.
A = { 3 1 6 2 1 3 4 5 9 0 }
Sorting
◻
Sorting = ordering.
◻
Sorted = ordered based on a particular way.
◻
Generally, collections of data are presented in a sorted
manner.
◻
Examples of Sorting:
Words in a dictionary are sorted (and case distinctions
are ignored).
Files in a directory are often listed in sorted order.
Sorting: Cont’d
Many banks provide statements that list checks
in increasing order (by check number).
In a newspaper, the calendar of events in a schedule is
generally sorted by date.
Musical compact disks in a record store are generally
sorted by recording artist.
◻
Why?
Types of Sorting Algorithms
◻
There are many, many different types of sorting
algorithms, but the primary ones are:
Bubble Sort
Selection Sort
Insertion Sort
Merge Sort
Quick Sort
Shell Sort Radix Sort
Swap Sort
Bubble Sort: Idea
◻
Idea: bubble in water.
Bubble in water moves upward. Why?
◻
How?
Bubble Sort Example
9, 6, 2, 12, 11, 9, 3, 7
6, 9, 2, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 11, 12, 9, 3, 7
6, 2, 9, 11, 9, 12, 3, 7
6, 2, 9, 11, 9, 3, 12, 7
6, 2, 9, 11, 9, 3, 7, 12
The 12 is greater than the 7 so they are exchanged. The 12 is greater than the 3 so they are exchanged. The twelve is greater than the 9 so they are exchanged The 12 is larger than the 11 so they are exchanged.
In the third comparison, the 9 is not larger than the 12 so no
exchange is made. We move on to compare the next pair without any change to the list.
Now the next pair of numbers are compared. Again the 9 is the larger and so this pair is also exchanged.
Bubblesort compares the numbers in pairs from left to right
exchanging when necessary. Here the first number is compared to the second and as it is larger they are exchanged.
Bubble Sort Example
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 11, 3, 7, 12
2, 6, 9, 9, 3, 11, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
Notice that this time we do not have to compare the last two
numbers as we know the 12 is in position. This pass therefore only requires 6 comparisons.
First Pass
Bubble Sort Example
2, 6, 9, 9, 3, 7, 11, 12
2, 6, 9, 3, 9, 7, 11, 12
2, 6, 9, 3, 7, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
Bubble Sort Example
2, 6, 9, 3, 7, 9, 11, 12
2, 6, 3, 9, 7, 9, 11, 12
2, 6, 3, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
Each pass requires fewer comparisons. This time only 4 are needed.
2, 6, 9, 3, 7, 9, 11, 12
Bubble Sort Example
2, 6, 3, 7, 9, 9, 11, 12
2, 3, 6, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
The list is now sorted but the algorithm does not know this until it completes a pass with no exchanges.
2, 6, 9, 3, 7, 9, 11, 12
Fourth Pass
2, 6, 3, 7, 9, 9, 11, 12
Bubble Sort Example
2, 3, 6, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
2, 6, 9, 3, 7, 9, 11, 12
Fourth Pass
2, 6, 3, 7, 9, 9, 11, 12
Fifth Pass
Sixth Pass
2, 3, 6, 7, 9, 9, 11, 12
Bubble Sort Example
Quiz Time
1.
Which number is definitely in its correct position at the
end of the first pass?
Answer: The last number must be the largest.
Answer: Each pass requires one fewer comparison than the last.
Answer: When a pass with no exchanges occurs.
2.
How does the number of comparisons required change as
the pass number increases?
3.
How does the algorithm know when the list is sorted?
4.
What is the maximum number of comparisons required
for a list of 10 numbers?
Bubble Sort: Example
40
2
1
43
3
65
0
-1
58
3
42
4
65
2
1
40
3
43
0
-1
58
3
42
4
65
58
1
2
3
40
0
-1
43
3
42
4
1
2
3
0
-1
40
3
42
4
43
58
65
1
2
3
4
◻
Notice that at least one element will be in the correct position
1
0
-1
2
3
3
4
40
42
43 58
65
Bubble Sort: Example
0
-1
1
2
3
3
4
40
42
43 58
65
-1
0
1
2
3
3
4
40
42
43 58
65
6
7
8
1
2
0
-1
3
3
40
4
42
43 58
65
Bubble Sort: Analysis
◻
Running time:
Worst case: O(N
2
)
Best case: O(N)
◻
Variant:
bi-directional bubble sort
Selection Sort: Idea
1.
We have two group of items:
sorted group, and
unsorted group
2.
Initially, all items are in the unsorted group. The
sorted group is empty.
We assume that items in the unsorted group
unsorted.
Selection Sort: Cont’d
1.
Select the “
best
” (eg. smallest) item from the
unsorted group, then put the “
best
” item at the
end of the sorted group.
2.
Repeat the process until the unsorted group
Selection Sort
5
1
3
4
6
2
Selection Sort
5
1
3
4
6
2
Selection Sort
5
1
3
4
6
2
Selection Sort
5
1
3
4
6
2
Selection Sort
5
1
3
4
6
2
Selection Sort
5
1
3
4
6
2
Selection Sort
5
1
3
4
6
2
Selection Sort
5
1
3
4
6
2
Comparison
Data Movement
Sorted
Selection Sort
5
1
3
4
2
6
Selection Sort
5
1
3
4
2
6
Selection Sort
5
1
3
4
2
6
Selection Sort
5
1
3
4
2
6
Selection Sort
5
1
3
4
2
6
Selection Sort
5
1
3
4
2
6
Selection Sort
5
1
3
4
2
6
Selection Sort
5
1
3
4
2
6
Comparison
Data Movement
Sorted
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Comparison
Data Movement
Sorted
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Comparison
Data Movement
Sorted
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Selection Sort
2
1
3
4
5
6
Comparison
Data Movement
Sorted
Selection Sort
1
2
3
4
5
6
Selection Sort
1
2
3
4
5
6
Comparison
Data Movement
Sorted
42
40
2
1
3
3
4
0
-1
43
58
65
40
2
1
43
3
4
0
-1
42
3
58
65
40
2
1
43
3
4
0
-1
58
3
42
65
40
2
1
43
3
65
0
-1
58
3
42
4
42
40
2
1
3
3
4
0
-1
43
58
65
42
-1
2
1
3
3
4
0
40
43
58
65
42
-1
2
1
3
3
0
4
40
43
58
65
42
-1
2
1
0
3
3
4
40
43
58
65
1
42
-1
2
1
0
3
3
4
40
43
58
65
42
-1
0
2
3
3
4
40
43
58
65
1
42
-1
0
2
3
3
4
40
43
58
65
1
42
0
2
3
3
4
40
43
58
65
-1
1
42
0
2
3
3
4
40
43
58
65
-1
Selection Sort: Analysis
◻
Running time:
Insertion Sort: Idea
◻
Idea: sorting cards.
8 | 5 9 2 6 3
5 8 | 9 2 6 3
5 8 9 | 2 6 3
2 5 8 9 | 6 3
2 5 6 8 9 | 3
Insertion Sort: Idea
1.
We have two group of items:
sorted group, and
unsorted group
2.
Initially, all items in the unsorted group and the sorted group
is empty.
We assume that items in the unsorted group unsorted.
We have to keep items in the sorted group sorted.
3.
Pick any item from, then insert the item at the right position
in the sorted group to maintain sorted property.
40
2
1
43
3
65
0
-1
58
3
42
4
2
40
1
43
3
65
0
-1
58
3
42
4
1
2
40 43
3
65
0
-1
58
3
42
4
40
1
2
3
40 43 65
0
-1
58
3
42
4
1
2
40 43
3
65
0
-1
58
3
42
4
1
2
3
40 43 65
0
-1
58
3
42
4
1
2
3
40 43 65
0
-1
58
3
42
4
1
2
3
40 43 65
0
-1
58
3
42
4
1
2
3
40 43 65
0
0
1
2
3
40 43 65
58
3
42
4
-1
1
2
3
40 43 65
0
0
1
2
3
40 43
58
65
3
42
4
-1
1
2
3
40 43 65
0
0
1
2
3
3
43
58
65
42
4
-1
40 43
58
65
1
2
3
40 43 65
0
0
1
2
3
3
43
42
65
4
-1
40
43
58
65
Insertion Sort: Example
1
2
3
40 43 65
0
0
1
2
3
3
43
42
65
Insertion Sort: Analysis
◻
Running time analysis:
A Lower Bound
◻
Bubble Sort, Selection Sort, Insertion Sort all have
worst case of O(N
2
).
◻
Turns out, for any algorithm that exchanges
adjacent items, this is the best worst case: Ω(N
2
)
Mergesort
◻
Mergesort (divide-and-conquer)
Divide array into two halves.
A
L
G
O
R
I
T
H
M
S
divide
Mergesort
◻
Mergesort (divide-and-conquer)
Divide array into two halves.
Recursively sort each half.
sort
A
L
G
O
R
I
T
H
M
S
divide
A
L
G
O
R
I
T
H
M
S
Mergesort
◻
Mergesort (divide-and-conquer)
Divide array into two halves.
Recursively sort each half.
Merge two halves to make sorted whole.
merge
sort
A
L
G
O
R
I
T
H
M
S
divide
A
L
G
O
R
I
T
H
M
S
A
G
L
O
R
H
I
M
S
T
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
A
Merging
◻Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
A
G
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
A
G
H
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
A
G
H
I
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
A
G
H
I
L
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
A
G
H
I
L
M
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
smallest
smallest
A
G
L
O
R
H
I
M
S
T
A
G
H
I
L
M
O
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
auxiliary array
first half
exhausted
smallest
A
G
L
O
R
H
I
M
S
T
A
G
H
I
L
M
O
R
Merging
◻
Merge.
Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
auxiliary array
first half
exhausted
smallest
A
G
L
O
R
H
I
M
S
T
A
G
H
I
L
M
O
R
S
Merging
◻
Merge.
Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.
Notes on Quicksort
◻
Quicksort is more widely used than any other sort.
◻
Quicksort is well-studied, not difficult to
implement, works well on a variety of data, and
consumes fewer resources that other sorts in nearly
all situations.
◻
Quicksort is O(n*log n) time, and O(log n)
Quicksort Algorithm
◻
Quicksort is a divide-and-conquer method for
sorting. It works by partitioning an array into parts,
then sorting each part independently.
◻
The crux of the problem is how to partition the
array such that the following conditions are true:
There is some element, a[i], where
a[i] is in its final position.
For all l < i, a[l] < a[i].
Quicksort Algorithm (cont)
◻
As is typical with a recursive program, once you figure
out how to divide your problem into smaller
subproblems, the implementation is amazingly simple.
int partition(Item a[], int l, int r);
void quicksort(Item a[], int l, int r)
{ int i;
if (r <= l) return;
i = partition(a, l, r);
quicksort(a, l, i-1);
quicksort(a, i+1, r);
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
Q
U
I
C
K
S
O
R
T
I
S
C
O
O
L
partitioned
partition
element
left
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
swap me
Q
U
I
C
K
S
O
R
T
I
S
C
O
O
L
partitioned
partition
element
left
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
Q
U
I
C
K
S
O
R
T
I
S
C
O
O
L
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
swap me
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
C
U
I
C
K
S
O
R
T
I
S
Q
O
O
L
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ exchange
■ repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ Exchange and repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ Exchange and repeat until pointers cross
swap me
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ Exchange and repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ Exchange and repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ Exchange and repeat until pointers cross
partitioned
partition
element
left
right
unpartitioned
swap me
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ Exchange and repeat until pointers cross
pointers cross
swap with
partitioning
element
partitioned
partition
element
left
right
unpartitioned
Partitioning in Quicksort
How do we partition the array efficiently?
■ choose partition element to be rightmost element
■ scan from left for larger element
■ scan from right for smaller element
■ Exchange and repeat until pointers cross
