Sorting Bubble Sort Selection Sort Insertion Sort Merge Sort Quick Sort

(1)

DATA STRUCTURES

AND ALGORITHMS

(2)



Sorting



Bubble Sort



Selection Sort



Insertion Sort



Merge Sort



Quick Sort

(3)

Sorting: Definition

Sorting: an operation that segregates

items into groups according to specified

criterion.

A = { 3 1 6 2 1 3 4 5 9 0 }

(4)

Sorting



Sorting = ordering.



Sorted = ordered based on a particular way.



Generally, collections of data are presented in a sorted

manner.



Examples of Sorting:



Words in a dictionary are sorted (and case distinctions

are ignored).



Files in a directory are often listed in sorted order.



The index of a book is sorted (and case distinctions are

(5)

Sorting: Cont’d



Many banks provide statements that list checks

in increasing order (by check number).



In a newspaper, the calendar of events in a schedule is

generally sorted by date.



Musical compact disks in a record store are generally

sorted by recording artist.



Why?



Imagine finding the phone number of your friend in

(6)

Types of Sorting Algorithms



There are many, many different types of sorting

algorithms, but the primary ones are:



Bubble Sort



Selection Sort



Insertion Sort



Merge Sort



Quick Sort



Shell Sort Radix Sort



Swap Sort

(7)

Bubble Sort: Idea



Idea: bubble in water.



Bubble in water moves upward. Why?



How?



When a bubble moves upward, the water from above

(8)

Bubble Sort Example

9, 6, 2, 12, 11, 9, 3, 7

6, 9, 2, 12, 11, 9, 3, 7

6, 2, 9, 12, 11, 9, 3, 7

6, 2, 9, 11, 12, 9, 3, 7

6, 2, 9, 11, 9, 12, 3, 7

6, 2, 9, 11, 9, 3, 12, 7

6, 2, 9, 11, 9, 3, 7, 12

The 12 is greater than the 7 so they are exchanged.

The 12 is greater than the 3 so they are exchanged.

The twelve is greater than the 9 so they are exchanged

The 12 is larger than the 11 so they are exchanged.

In the third comparison, the 9 is not larger than the 12 so no

exchange is made. We move on to compare the next pair without any change to the list.

In the third comparison, the 9 is not larger than the 12 so no

exchange is made. We move on to compare the next pair without any change to the list.

Now the next pair of numbers are compared. Again the 9 is the larger and so this pair is also exchanged.

Bubblesort compares the numbers in pairs from left to right

exchanging when necessary. Here the first number is compared to the second and as it is larger they are exchanged.

Bubblesort compares the numbers in pairs from left to right

exchanging when necessary. Here the first number is compared to the second and as it is larger they are exchanged.

The end of the list has been reached so this is the end of the first pass. The twelve at the end of the list must be largest number in the list and so is now in the correct position. We now start a new pass from left to right.

(9)

Bubble Sort Example

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 11, 3, 7, 12

2, 6, 9, 9, 3, 11, 7, 12

2, 6, 9, 9, 3, 7, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

Notice that this time we do not have to compare the last two

numbers as we know the 12 is in position. This pass therefore only requires 6 comparisons.

Notice that this time we do not have to compare the last two

numbers as we know the 12 is in position. This pass therefore only requires 6 comparisons.

First Pass

(10)

Bubble Sort Example

2, 6, 9, 9, 3, 7, 11, 12

2, 6, 9, 3, 9, 7, 11, 12

2, 6, 9, 3, 7, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

This time the 11 and 12 are in position. This pass therefore only requires 5 comparisons.

(11)

Bubble Sort Example

2, 6, 9, 3, 7, 9, 11, 12

2, 6, 3, 9, 7, 9, 11, 12

2, 6, 3, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

Each pass requires fewer comparisons. This time only 4 are needed.

(12)

Bubble Sort Example

2, 6, 3, 7, 9, 9, 11, 12

2, 3, 6, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

The list is now sorted but the algorithm does not know this until it completes a pass with no exchanges.

2, 6, 9, 3, 7, 9, 11, 12

Fourth Pass

(13)

Bubble Sort Example

2, 3, 6, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12

Second Pass

First Pass

Third Pass

2, 6, 9, 3, 7, 9, 11, 12

Fourth Pass

2, 6, 3, 7, 9, 9, 11, 12

Fifth Pass

Sixth Pass

2, 3, 6, 7, 9, 9, 11, 12

This pass no exchanges are made so the algorithm knows the list is sorted. It can therefore save time by not doing the final pass. With other lists this check could save much more work.

(14)

Bubble Sort Example

Quiz Time

1. Which number is definitely in its correct position at the

end of the first pass?

Answer: The last number must be the largest.

Answer: Each pass requires one fewer comparison than the last.

Answer: When a pass with no exchanges occurs.

2. How does the number of comparisons required change as

the pass number increases?

3. How does the algorithm know when the list is sorted?

4. What is the maximum number of comparisons required

for a list of 10 numbers?

(15)

Bubble Sort: Example

40

2

1

43

3

65

0 -1

58

3

42

4

65

2

1

40

3

43

0 -1

58

3

42

4

65

58

1

2

3

40

0 -1

43

3

42

4

1

2

3

0 -1

40

3

42

4

43

58

65

1

2

3

4



Notice that at least one element will be in the correct position

(16)

1

0 -1

2

3

4

40

42 43 58

65 Bubble Sort: Example

0 -1

1

2

3

4

40

42 43 58

65 -1

0

1

2

3

4

40

42 43 58

65

6

7

8

1

2

0 -1

3

40

4

42 43 58

65

(17)

(18)

Bubble Sort: Analysis



Running time:



Worst case: O(N

2

)



Best case: O(N)



Variant:



bi-directional bubble sort



original bubble sort: only works to one direction

(19)

Selection Sort: Idea

1. We have two group of items:



sorted group, and



unsorted group

2. Initially, all items are in the unsorted group. The

sorted group is empty.



We assume that items in the unsorted group

unsorted.

(20)

Selection Sort: Cont’d

1. Select the “

best

” (eg. smallest) item from the

unsorted group, then put the “

best

” item at the

end of the sorted group.

2. Repeat the process until the unsorted group

(21)

Selection Sort

5

1

3

4

6

2

(22)

Selection Sort

5

1

3

4

6

2

(23)

Selection Sort

5

1

3

4

6

2

(24)

Selection Sort

5

1

3

4

6

2

(25)

Selection Sort

5

1

3

4

6

2

(26)

Selection Sort

5

1

3

4

6

2

(27)

Selection Sort

5

1

3

4

6

2

(28)

Selection Sort

5

1

3

4

6

2 Comparison

Data Movement

Sorted



(29)

Selection Sort

5

1

3

4

2

6

(30)

Selection Sort

5

1

3

4

2

6

(31)

Selection Sort

5

1

3

4

2

6

(32)

Selection Sort

5

1

3

4

2

6

(33)

Selection Sort

5

1

3

4

2

6

(34)

Selection Sort

5

1

3

4

2

6

(35)

Selection Sort

5

1

3

4

2

6

(36)

Selection Sort

5

1

3

4

2

6 Comparison

Data Movement

Sorted



(37)

Selection Sort

2

1

3

4

5

6

(38)

Selection Sort

2

1

3

4

5

6

(39)

Selection Sort

2

1

3

4

5

6

(40)

Selection Sort

2

1

3

4

5

6

(41)

Selection Sort

2

1

3

4

5

6

(42)

Selection Sort

2

1

3

4

5

6

(43)

Selection Sort

2

1

3

4

5

6 Comparison

Data Movement

Sorted



(44)

Selection Sort

2

1

3

4

5

6

(45)

Selection Sort

2

1

3

4

5

6

(46)

Selection Sort

2

1

3

4

5

6

(47)

Selection Sort

2

1

3

4

5

6

(48)

Selection Sort

2

1

3

4

5

6

(49)

Selection Sort

2

1

3

4

5

6 Comparison

Data Movement

Sorted



(50)

Selection Sort

2

1

3

4

5

6

(51)

Selection Sort

2

1

3

4

5

6

(52)

Selection Sort

2

1

3

4

5

6

(53)

Selection Sort

2

1

3

4

5

6

(54)

Selection Sort

2

1

3

4

5

6 Comparison

Data Movement

Sorted



(55)

Selection Sort

1

2

3

4

5

6

(56)

Selection Sort

1

2

3

4

5

6 Comparison

Data Movement

Sorted

(57)

42

40

2

1

3

4

0 -1

43

58

65

40

2

1

43

3

4

0 -1 42

3

58

65

40

2

1

43

3

4

0 -1 58

3

42

65

40

2

1

43

3

65

0 -1 58

3

42

4

(58)

42

40

2

1

3

4

0 -1

43

58

65

42 -1

2

1

3

4

0

40

43

58

65

42 -1

2

1

3

0

4

40

43

58

65

42 -1

2

1

0

3

4

40

43

58

65

(59)

1

42 -1

2

1

0

3

4

40

43

58

65

42 -1

0

2

3

4

40

43

58

65

1

42 -1

0

2

3

4

40

43

58

65

1

42

0

2

3

4

40

43

58

65 -1

1

42

0

2

3

4

40

43

58

65 -1

(60)

(61)

Selection Sort: Analysis



Running time:



Worst case: O(N

2

)

(62)

Insertion Sort: Idea



Idea: sorting cards.



_{8 | 5 9 2 6 3}



5 8 | 9 2 6 3



5 8 9 | 2 6 3



2 5 8 9 | 6 3



2 5 6 8 9 | 3

(63)

Insertion Sort: Idea

1.

We have two group of items:



sorted group, and



unsorted group

2.

Initially, all items in the unsorted group and the sorted group

is empty.



We assume that items in the unsorted group unsorted.



We have to keep items in the sorted group sorted.

3.

Pick any item from, then insert the item at the right position

in the sorted group to maintain sorted property.

(64)

40

2

1

43

3

65

0 -1 58

3

42

4

2

40

1

43

3

65

0 -1 58

3

42

4

1

2 40 43

3

65

0 -1 58

3

42

4

40

(65)

1

2

3 40 43 65

0 -1 58

3

42

4

1

2 40 43

3

65

0 -1 58

3

42

4

1

2

3 40 43 65

0 -1 58

3

42

4

(66)

1

2

3 40 43 65

0 -1 58

3

42

4

1

2

3 40 43 65

0 -1 58

3

42

4

1

2

3 40 43 65

0

1

2

3 40 43 65

58

3

42

4 -1

(67)

1

2

3 40 43 65

0

1

2

3 40 43

58

65

3

42

4 -1

1

2

3 40 43 65

0

1

2

3

43

58

65

42

4 -1

40 43

58

65

1

2

3 40 43 65

0

1

2

3

43

42

65

4 -1

40

43

58

65 Insertion Sort: Example

1

2

3 40 43 65

0

1

2

3

43

42

65

(68)

(69)

Insertion Sort: Analysis



Running time analysis:



Worst case: O(N

2

)

(70)

A Lower Bound



Bubble Sort, Selection Sort, Insertion Sort all have

worst case of O(N

2 ).



Turns out, for any algorithm that exchanges

adjacent items, this is the best worst case: Ω(N

2 )

(71)

Mergesort



Mergesort (divide-and-conquer)



Divide array into two halves.

A

L

G

O

R

I

T

H

M

S

divide

(72)

Mergesort



Mergesort (divide-and-conquer)



Divide array into two halves.



Recursively sort each half.

sort

A

L

G

O

R

I

T

H

M

S

divide

A

L

G

O

R

I

T

H

M

S

(73)

Mergesort



Mergesort (divide-and-conquer)



Divide array into two halves.



Recursively sort each half.



Merge two halves to make sorted whole.

merge

sort

A

L

G

O

R

I

T

H

M

S

divide

A

L

G

O

R

I

T

H

M

S

A

G

L

O

R

H

I

M

S

T

(74)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

Merging



Merge.

 _{Keep track of smallest element in each sorted half.}  _{Insert smallest of two elements into auxiliary array.}  Repeat until done.

(75)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

A

Merging

Merge.

 Keep track of smallest element in each sorted half.  Insert smallest of two elements into auxiliary array.  Repeat until done.

(76)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

A

G

Merging



Merge.

(77)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

A

G

H

Merging



Merge.

(78)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

A

G

H

I

Merging



Merge.

(79)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

A

G

H

I

L

Merging



Merge.

(80)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

A

G

H

I

L

M

Merging



Merge.

(81)

auxiliary array

smallest

A

G

L

O

R

H

I

M

S

T

A

G

H

I

L

M

O

Merging



Merge.

(82)

auxiliary array

first half

exhausted

_smallest

A

G

L

O

R

H

I

M

S

T

A

G

H

I

L

M

O

R

Merging



Merge.



Keep track of smallest element in each sorted half.



Insert smallest of two elements into auxiliary array.



Repeat until done.

(83)

auxiliary array

first half

exhausted

_smallest

A

G

L

O

R

H

I

M

S

T

A

G

H

I

L

M

O

R

S

Merging



Merge.

(84)

Notes on Quicksort



Quicksort is more widely used than any other sort.



Quicksort is well-studied, not difficult to

implement, works well on a variety of data, and

consumes fewer resources that other sorts in nearly

all situations.



Quicksort is O(n*log n) time, and O(log n)

(85)

Quicksort Algorithm



Quicksort is a divide-and-conquer method for

sorting. It works by partitioning an array into

parts, then sorting each part independently.



The crux of the problem is how to partition the

array such that the following conditions are true:



_{There is some element, a[i], where}

a[i] is in its final position.



_{For all l < i, a[l] < a[i].}

(86)

Quicksort Algorithm (cont)



As is typical with a recursive program, once you figure out how to

divide your problem into smaller subproblems, the implementation

is amazingly simple.

int partition(Item a[], int l, int r);

void quicksort(Item a[], int l, int r)

{ int i;

if (r <= l) return;

(87)

(88)

(89)

Partitioning in Quicksort



How do we partition the array efficiently?

 _{choose partition element to be rightmost element}  _{scan from left for larger element}

 _{scan from right for smaller element}  _exchange

 _{repeat until pointers cross}

Q

U

I

C

K

S

O

R

T

I

S

C

O

L

partitioned

partition element

left

(90)

Partitioning in Quicksort



How do we partition the array efficiently?

swap me

Q

U

I

C

K

S

O

R

T

I

S

C

O

L

partitioned

partition element

left

(91)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

swap me

(92)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

swap me

(93)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

swap me

Q

U

I

C

K

S

O

R

T

I

S

C

O

L

(94)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

(95)

Partitioning in Quicksort



How do we partition the array efficiently?

swap me

partitioned

partition element

left

right

unpartitioned

(96)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

swap me

(97)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

swap me

C

U

I

C

K

S

O

R

T

I

S

Q

O

L

(98)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

(99)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

(100)

Partitioning in Quicksort



How do we partition the array efficiently?

partitioned

partition element

left

right

unpartitioned

(101)

Partitioning in Quicksort



How do we partition the array efficiently?

 _{scan from right for smaller element}  _{Exchange and repeat until pointers cross}