Exercises
2.2 SUMS AND RECURRENCES
OK, we understand now how to express sums with fancy notation. But how does a person actually go about nding the value of a sum? One way is to observe that there's an intimate relation between sums and recurrences. The sum Sn = n X k=0 ak
is equivalent to the recurrence
(Think of Sn as
not just a single number, but as a sequence dened for all n0.)
S0=a0;
Sn =Sn−1+an, forn > 0. (2.6)
Therefore we can evaluate sums in closed form by using the methods we learned in Chapter 1 to solve recurrences in closed form.
For example, ifan is equal to a constant plus a multiple of n, the sum-
recurrence (2.6) takes the following general form:
R0=α;
Rn=Rn−1+β+γn , forn > 0. (2.7)
Proceeding as in Chapter 1, we ndR1=α+β+γ, R2=α+2β+3γ, and
so on; in general the solution can be written in the form
whereA(n),B(n), andC(n)are the coecients of dependence on the general parametersα,β, andγ.
The repertoire method tells us to try plugging in simple functions ofn
forRn, hoping to nd constant parametersα,β, andγwhere the solution is
especially simple. SettingRn=1impliesα=1, β=0, γ=0; hence A(n) = 1 .
SettingRn=nimplies α=0,β=1,γ=0; hence B(n) = n .
SettingRn=n2impliesα=0,β= −1,γ=2; hence 2C(n) −B(n) = n2
and we haveC(n) = (n2+n)/2. Easy as pie. Actually easier;π= P
n0 8
(4n+1)(4n+3). Therefore if we wish to evaluate
n
X
k=0
(a+bk),
the sum-recurrence (2.6) boils down to (2.7) withα=β=a,γ=b, and the answer isaA(n) +aB(n) +bC(n) =a(n+1) +b(n+1)n/2.
Conversely, many recurrences can be reduced to sums; therefore the spe- cial methods for evaluating sums that we'll be learning later in this chapter will help us solve recurrences that might otherwise be dicult. The Tower of Hanoi recurrence is a case in point:
T0 = 0;
Tn = 2Tn−1+1 , forn > 0.
It can be put into the special form (2.6) if we divide both sides by2n: T0/20 = 0;
Tn/2n = Tn−1/2n−1+1/2n, forn > 0.
Now we can setSn=Tn/2n, and we have S0 = 0; Sn = Sn−1+2−n, forn > 0. It follows that Sn = n X k=1 2−k.
(Notice that we've left the term fork=0 out of this sum.) The sum of the geometric series2−1+2−2+· · ·+2−n= (1
2)1+(12)2+· · ·+(12)nwill be derived
later in this chapter; it turns out to be1− (1
2)n. HenceTn=2nSn=2n−1.
We have converted Tn to Sn in this derivation by noticing that the re-
currence could be divided by 2n. This trick is a special case of a general
technique that can reduce virtually any recurrence of the form
anTn = bnTn−1+cn (2.9)
to a sum. The idea is to multiply both sides by a summation factor,sn: snanTn = snbnTn−1+sncn.
This factorsn is cleverly chosen to make snbn = sn−1an−1.
Then if we writeSn =snanTn we have a sum-recurrence, Sn = Sn−1+sncn. Hence Sn = s0a0T0+ n X k=1 skck = s1b1T0+ n X k=1 skck,
and the solution to the original recurrence (2.9) is
Tn = 1 snan µ s1b1T0+ n X k=1 skck ¶ . (2.10)
For example, whenn=1we getT1= (s1b1T0+s1c1)/s1a1= (b1T0+c1)/a1. (The value ofs1
cancels out, so it can be anything but zero.)
But how can we be clever enough to nd the rightsn? No problem: The
relationsn =sn−1an−1/bn can be unfolded to tell us that the fraction sn = an−1an−2. . . a1
bnbn−1. . . b2
, (2.11)
or any convenient constant multiple of this value, will be a suitable summation factor. For example, the Tower of Hanoi recurrence hasan =1 andbn =2;
the general method we've just derived says thatsn =2−n is a good thing to
multiply by, if we want to reduce the recurrence to a sum. We don't need a brilliant ash of inspiration to discover this multiplier.
We must be careful, as always, not to divide by zero. The summation- factor method works whenever all thea's and all theb's are nonzero.
Let's apply these ideas to a recurrence that arises in the study of \quick-
sort," one of the most important methods for sorting data inside a computer. (Quicksort was invented by Hoare in 1962 [189].)
The average number of comparison steps made by quicksort when it is applied tonitems in random order satises the recurrence
C0=0; Cn=n+1+ 2 n nX−1 k=0 Ck, forn > 0. (2.12) Hmmm. This looks much scarier than the recurrences we've seen before; it includes a sum over all previous values, and a division by n. Trying small cases gives us some data (C1=2,C2=5,C3= 263) but doesn't do anything
to quell our fears.
We can, however, reduce the complexity of (2.12) systematically, by rst getting rid of the division and then getting rid of thePsign. The idea is to multiply both sides byn, obtaining the relation
nCn = n2+n+2 nX−1 k=0 Ck, forn > 0; hence, if we replacenbyn−1, (n−1)Cn−1 = (n−1)2+ (n−1) +2 nX−2 k=0 Ck, forn−1 > 0.
We can now subtract the second equation from the rst, and the P sign disappears:
nCn− (n−1)Cn−1 = 2n+2Cn−1, forn > 1.
It turns out that this relation also holds when n = 1, because C1 = 2.
Therefore the original recurrence forCn reduces to a much simpler one: C0 = 0;
nCn = (n+1)Cn−1+2n , forn > 0.
Progress. We're now in a position to apply a summation factor, since this recurrence has the form of (2.9) with an = n, bn = n+1, and cn = 2n.
The general method described on the preceding page tells us to multiply the recurrence through by some multiple of
sn = an−1an−2. . . a1 bnbn−1. . . b2 = (n−1)·(n−2)·. . .·1 (n+1)·n·. . .·3 = 2 (n+1)n.
The solution, according to (2.10), is therefore
We started with a P
in the recur- rence, and worked hard to get rid of it. But then after ap- plying a summation factor, we came up with another P. Are sums good, or bad, or what? Cn = 2(n+1) n X k=1 1 k+1.
The sum that remains is very similar to a quantity that arises frequently in applications. It arises so often, in fact, that we give it a special name and a special notation: Hn = 1+1 2 +· · ·+ 1 n = n X k=1 1 k. (2.13)
The letter H stands for \harmonic"; Hn is a harmonic number, so called
because the kth harmonic produced by a violin string is the fundamental tone produced by a string that is1/ktimes as long.
We can complete our study of the quicksort recurrence (2.12) by putting
Cn into closed form; this will be possible if we can express Cn in terms of Hn. The sum in our formula for Cn is
n X k=1 1 k+1 = X 1kn 1 k+1.
We can relate this toHn without much diculty by changingkto k−1 and
revising the boundary conditions:
X 1kn 1 k+1 = X 1k−1n 1 k = X 2kn+1 1 k = µ X 1kn 1 k ¶ −1 1 + 1 n+1 = Hn− n n+1.
Alright! We have found the sum needed to complete the solution to (2.12):
But your spelling is
alwrong. The average number of comparisons made by quicksort when it is applied to nrandomly ordered items of data is
Cn = 2(n+1)Hn−2n . (2.14)