Design and Analysis of
Algorithms
Designing Algorithms
• There are many ways to design algorithms
• Insertion sort uses an incremental approach
– Having sorted to array A[i..j]
– We insert the single element A[j] into its proper place, to get a sorted array A[i..j]
• An alternative design approach
•
Divide
the problems into a number of sub
problems.
•
Conquer
the sub problems by solving
them recursively. If the sub-problem sizes
are small enough, just solve the problems
in a straight forward manner.
•
Combine
the solutions to the sub
problems into the solution for the original
problem.
•
Divide
the n element sequence to be
sorted into two subsequences of n/2
elements each.
•
Conquer:
Sort the two subsequences to
produce the sorted answer.
•
Combine:
Merge the two sorted sub
Merge Sort
Base Case: When the sequences to be sorted has length 1.
108 56 14 89 12 34 66
Unsorted
108 56 14 66 Divide 108 66 Divide Merge 56 14 Divide 14 56 Merge 56 66 108 14 Merge 12 89 34 Divide 12 89 Divide 89 12 Merge 34 Divide 34 BCase 34 Merge 34 12 89 Merge 14 34 56 66 89 108 12
Merge-sort(A, f, l)
1.
if
f < l
2.
then
m = (f + l)/2
3.
Merge-sort(A, f, m)
4.
Merge-sort(A, m + 1, l)
5. Merge(A, f, m, l)
Merge(A, f, m, l)
1. T[f..l] \\declare temporary array of same size
2. i f; k f; j m + 1 \\initialize integers i, j, and k
3. while (i m) and (j l)
4. do if (A[i] A[j]) \\comparison of elements
5. then T[k++] A[i++] 6. else T[k++] A[j++] 7. while (i m)
8. do T[k++] A[i++] \\copy from A to T
Merge Algorithm
• The basic merging algorithms takes
– Two input arrays, A[] and B[],
– One output array C[]
– And three counters aptr, bptr and cptr. (initially
set to the beginning of their respective arrays)
• The smaller of A[aptr] and B[bptr] is copied to the
next entry in C i.e. C[cptr].
• The appropriate counters are then advanced.
• When either of input list is exhausted, the
Merge Algorithm
56 66 108 14
A[]
aptr
34
12 89
B[]
bptr
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
14 > 12 therefore
C[]
Merge Algorithm
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
14 > 12 therefore
cptr++ C[]
bptr
C[cptr] = 12
Merge Algorithm
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
14 > 12 therefore
cptr++ C[]
bptr
bptr++
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
14 < 34 therefore
C[]
bptr
12
C[cptr] = 14
14
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
14 < 34 therefore
cptr++ C[]
bptr
C[cptr] = 14
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
14 < 34 therefore
cptr++ C[]
bptr aptr
aptr++
12
C[cptr] = 14
14
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
56 > 34 therefore
C[]
bptr
C[cptr] = 34
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
56 > 34 therefore
cptr++ C[]
bptr
12
C[cptr] = 34
14 34
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
56 > 34 therefore
cptr++ C[]
bptr
bptr++
C[cptr] = 34
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
56 < 89 therefore
C[]
bptr
12 14 34
C[cptr] = 56
56
56 66 108 14
A[]
aptr
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
56 < 89 therefore
cptr++ C[]
bptr
C[cptr] = 56
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
56 < 89 therefore
cptr++ C[]
bptr aptr
aptr++
12 14 34
C[cptr] = 56
56
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
66 < 89 therefore
C[]
bptr aptr
C[cptr] = 66
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
66 < 89 therefore
cptr++ C[]
bptr aptr
12 14 34
C[cptr] = 66
56 66
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
66 < 89 therefore
cptr++ C[]
bptr aptr
aptr++
C[cptr] = 66
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
108 > 89 therefore
C[]
bptr aptr
12 14 34 56 66 C[cptr] = 89
89
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
108 > 89 therefore
cptr++ C[]
bptr aptr
C[cptr] = 89
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
108 > 89 therefore
cptr++ C[]
bptr aptr
bptr++
12 14 34 56 66 C[cptr] = 89
89
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
Array B is now finished, copy
remaining elements of array A in array C
C[]
bptr aptr
56 66 108 14
A[]
34
12 89
B[]
If A[aptr] < B[bptr]
C[cptr++] = A[aptr++]
Else
C[cptr++] = B[bptr++]
Array B is now finished, copy
remaining elements of array A in array C
C[]
bptr aptr
12 14 34 56 66 89 108
• Let T(n) be the time taken by this algorithm to sort an array of n
elements dividing A into sub-arrays A1and A2.
• It is easy to see that the Merge (A1, A2, A) takes the linear time. Consequently,
T(n) = T(n/2) + T(n/2) + θ(n)
T(n) = 2T (n/2) + θ(n)
• The above recurrence relation is non-homogenous and can be solved by any of the methods
– Defining characteristics polynomial
n
n
T
n
T
)
2
(
.
2
)
(
2
)
2
(
.
2
)
2
(
n
T
n
2n
T
2 3
2
)
2
.
(
2
)
2
2
(
n
T
n
n
T
. . . 2 ) 2 ( . 2 ) 2
( n3 T n4 n3
T
n n
n
n
n
n
T
n
n
T
n
T
)
2
(
.
2
)
(
)
2
(
.
2
)
(
2 2n
n
n
n
T
n
T
)
2
(
.
2
)
(
3 3n
n
n
T
n
T
)
2
(
.
2
)
(
2 2.
.
.
• T(n) = 2T(n/2) + n
• = 2(2T(n/4) + n/2) + n = 4T(n/4) + n + n = 4T(n/4) + 2n
• = 4(2T(n/8) + n/4) + 2n = 8T(n/8) + n + 2n = 8T(n/8) + 3n
• = 8(2T(n/16) + n/8)+ 3n = 8T(n/16)+ n + 3n = 16T(n/16) + 4n
• ... = 32T(n/32) + 5n
• ...
• = n*T(1) + log2(n)*n
