Recursion - Recurrence Relations.pptx

Recursion and

Recurrence Relations

• _{Definition: A recurrence relation for a sequence {an} is an equation that}

expresses a_n in terms of one or more of the previous terms in the sequence: a₀, a₁, a₂, …, a_n-1

for all integers nn₀ where n₀ is a nonnegative integer. OR simply

A recurrence relation is an equation which is defined in terms of itself.

• _{A sequence is called a solution of a recurrence if its terms satisfy the}

Fall 2002 CMSC 203 - Discrete Structures 3

Recurrence Relations

•_{In other words, a recurrence relation is like a recursively}

defined sequence, but without specifying any initial values (initial conditions).

•_{Therefore, the same recurrence relation can have (and}

usually has) multiple solutions.

•_{If both the initial conditions and the recurrence relation}

Fall 2002 CMSC 203 - Discrete Structures 4

Recurrence Relations

Consider the recurrence relation a_n = 2a_n-1 – a_n-2 for n = 2, 3, 4, …

•_{Is the sequence {an} with an=3n a solution of this}

recurrence relation?

•_{For n}  2 we see that

2a_n-1 – a_n-2 = 2(3(n – 1)) – 3(n – 2) = 3n = a_n.

•_{Therefore, {an} with an=3n is a solution of the recurrence}

Fall 2002 CMSC 203 - Discrete Structures 5

Recurrence Relations

•_{Is the sequence {an} with an= 5 a solution of the same}

recurrence relation?

•_{For n}  _{2 we see that}

2a_n-1 – a_n-2 = 25 - 5 = 5 = a_n.

•_{Therefore, {an} with an=5 is also a solution of the}

More examples of recurrence

relations

• Example 1:

• Initial condition a₀ = 1

• Recursive formula: a _n = 1 + 2a _n-1 for n > 1 • First few terms are: 1, 3, 7, 15, 31, 63, …

• Example 2:

• Initial conditions a₀ = 1, a₁ = 2

• Recursive formula: a _n = 3(a _n-1 + a _n-2) for n > 2

More examples of recurrence relations

• _{The Fibonacci numbers are defined by the recurrence}

relation

F(n) = F(n-1) +F(n-2) F(1) = 1

F(0) = 0

• _{The solution to the Fibonacci recurrence is}

Recurrence Relations: Initial Conditions

• The initial conditions specify the value of the first few necessary terms in the sequence

• In the Fibonacci numbers, we needed two initial conditions: F(0)=0 and F(1)=1

because F(n) is defined by the two previous terms in the sequence

• Initial conditions are also known as boundary conditions (as opposed to general conditions)

• From now on, we will use the subscript notation, so the Fibonacci numbers are:

f_n = f_n-1 + f_n-2 f₁ = 1

• _{Find the Fibonacci numbers f2, f3, f4, f5, and f6.}

• _{Solution: The recurrence relation for the Fibonacci sequence tells us that we find}

successive terms by adding the previous two terms. Because the initial conditions tell us that f₀ = 0 and f₁ = 1, using the recurrence relation in the definition we find that

10

Fibonacci sequence in nature

11

Reproducing rabbits

• _{You have one pair of rabbits on an island}

• _{The rabbits repeat the following:}

• Get pregnant one month

• Give birth (to another pair) the next month

• _{This process repeats indefinitely (no deaths)} • _{Rabbits get pregnant the month they are born}

12

Reproducing rabbits

• _{First month: 1 pair}

• _{The original pair}

• _{Second month: 1 pair}

• _{The original (and now pregnant) pair}

• _{Third month: 2 pairs}

• _{The child pair (which is pregnant) and the parent pair (recovering)}

• _{Fourth month: 3 pairs}

• _{“Grandchildren”: Children from the baby pair (now pregnant)} • _{Child pair (recovering)}

• _{Parent pair (pregnant)}

• _{Fifth month: 5 pairs}

• _{Both the grandchildren and the parents reproduced}

13

Reproducing rabbits

• _{Sixth month: 8 pairs}

• _{All 3 new rabbit pairs are pregnant, as well as those not pregnant in the last month (2)}

• _{Seventh month: 13 pairs}

• All 5 new rabbit pairs are pregnant, as well as those not pregnant in the last month (3)

• _{Eighth month: 21 pairs}

• All 8 new rabbit pairs are pregnant, as well as those not pregnant in the last month (5)

• _{Ninth month: 34 pairs}

• _{All 13 new rabbit pairs are pregnant, as well as those not pregnant in the last month (8)}

• _{Tenth month: 55 pairs}

• _{The Fibonacci numbers appear in many other places in nature,}

15

Fibonacci sequence

• As the terms increase, the ratio between successive terms approaches 1.618

• This is called the “golden ratio”

• _{Ratio of human leg length to arm length} • _{Ratio of successive layers in a conch shell}

• _{Reference: http://en.wikipedia.org/wiki/Golden_ratio}

618933989 . 1 2 1 5 ) ( ) 1 (

lim     



 F n 

n F

Recurrence Relations: Terms

• _{Recurrence relations have two parts:}

• recursive terms and

• non-recursive terms

T(n) = 2T(n-2) + n2 _-10

• _{Recursive terms come from when an algorithms calls itself}

• Non-recursive terms correspond to the non-recursive cost of the algorithm: work the algorithm performs within a function

Solving Recurrences

• _{There are several methods for solving recurrences}

• _{Characteristic Equations} • _{Forward Substitution} • _{Backward Substitution} • _{Recurrence Trees}

• _{… Maple!}

Method 1: Iteration

• Problem: Given a recursive expression with initial conditions a₀, a₁

• try to express a_n without dependence on

previous terms.

Example: a_n = 2a_n-1 for n > 1, with initial conditiona₀ = 1

More on the iteration method

Example: Deer Population growth

• Deer population d_n at time n • Initial condition: d₀ = 1000

• Increase from time n-1 to time n is 10%. Therefore the recursive function is

d_n – d_n-1 = 0.1d_n-1  d_n = 1.1d_n-1

Linear Homogeneous Recurrences

• Definition: A linear homogeneous recurrence relation of degree k with constant coefficients is a recurrence relation of the form

a_n = c₁a_n-1 + c₂a_n-2 + … + c_ka_n-k

with c₁, c₂, …, c_kR, c_k 0.

• Linear: RHS is a sum of multiples of previous terms of the sequence each multiplied by a function of n (linear combination of previous terms). The coefficients are all constants (not functions depending on n)

• _{Homogeneous: no terms occur that are not multiples of aj’s}

• _{Degree k: an is expressed in terms of (n-k)}th _{term of the sequence}

Linear homogeneous

recurrences-examples

• _{EXAMPLE 1 The recurrence relation Pn = (1.11)Pn−1 is a linear}

homogeneous recurrence relation of degree one. The recurrence relation f_n = f_n−1 + f_n−2 is a linear homogeneous recurrence relation of degree two. The recurrence relation a_n = a_n-5 is a linear homogeneous recurrence relation of degree five.

• _{EXAMPLE 2 The recurrence relation an = an−1 + a}2

Linear Homogeneous Recurrences: Examples

So are the following relations:

a_n = 4a_n-1 + 5a_n-2 + 7a_n-3 a_n = 2a_n-2 + 4a_n-4 + 8a_n-8

How many initial conditions do we need to specify for these relations?

• _{So, how do solve linear homogeneous recurrences?}

Solving Linear Homogeneous Recurrences

• _{We want a solution of the form an=r}n _{where r is some real constant}

• _{We observe that an=r}n _{is a solution to a linear homogeneous recurrence if and only if}

rn_{= c}

1rn-1+ c2rn-2+ … + ckrn-k

• We can now divide both sides by rn-k_{, collect terms and we get a k-degree polynomial}

rk_{- c}

1rk-1- c2rk-2- … - ck = 0

• This equation is called the characteristic equation of the recurrence relation

• The roots of this polynomial are called the characteristics roots of the recurrence relation. They can be used to find the solutions (if they exist) to the recurrence relation. We will consider

Second Order Linear Homogeneous Recurrences

• _{A second order (k=2) linear homogeneous recurrence is a recurrence of the form}

a_n = c₁a_n-1+ c₂a_n-2

• _{Theorem 1: Let c1, c2R and suppose that r}2_-c

1r-c2=0 is the characteristic

polynomial of a 2nd _{order linear homogeneous recurrence that has two distinct*}

roots r₁,r₂, then {a_n} is a solution if and only if

a_n= ₁r₁n ₊  2r2n

for n=0,1,2,… where ₁, ₂ are constants dependent upon the initial conditions

• Proof: We must do two things to prove the theorem. First, it must be shown that if r₁ and r₂ are the roots of the characteristic equation, and α₁ and α₂

are constants, then the sequence {a_n} with a_n = α₁r₁n_{+ α}

2r2n is a solution of

the recurrence relation. Second, it must be shown that if the sequence {a_n} is a solution, then a_n = α₁r₁n_{+ α}

2r2n for some constants α1 and α2.

• _{Now we will show that if an = α1r1}n_{+ α}

2r2n , then the sequence {an} is a

solution of the recurrence relation. Because r₁ and r₂ are roots of

r2 _{− c}

1r − c2 = 0, it follows that r12= c1r1 + c2,

r₂2_{= c}

• _{From these equations, we see that} • _{c1an−1 + c2an−2 = c1 (α1r1}n-1_{+ α}

2r2n-1 ) + c2 (α1r1n-2+ α2r2n-2 )

• _{= α1r1}n-2 _(c

1r1 + c2 ) + α2r2n-2 (c1r2 + c2 )

• _{= α1r1}n-2_r

12 + α2r2n-2 r22

• _{= α1r1}n _{+ α}

2r2n

• _{= an}

• _{This shows that the sequence {an} with an = α1r1}n _{+ α}

2r2n is a solution of

• _{To show that every solution {an} of the recurrence relation an = c1 an-1 +}

c₂ a_{n -2} has a_n = α₁r₁n _{+ α}

2r2n for n = 0, 1, 2, . . . , for some constants α1

and α₂, suppose that {a_n} is a solution of the recurrence relation, and the initial conditions a₀ = C₀ and a₁ = C₁ hold. It will be shown that there are constants α₁ and α₂ such that the sequence {a_n} with a_n =

α₁r₁n _{+ α}

2r2n satisfies these same initial conditions. This requires that

• _{a0 = C0 = α1 + α2,}

• _{We can solve these two equations for α1 and α2. From the first}

equation it follows that α₂ = C₀ − α₁. Inserting this expression into the second equation gives C₁ = α₁ r₁ + (C₀ − α₁) r₂.

• _{Hence, C1 = α1(r1 − r2) + C0 r2. This shows that α1 = and}

• _{We can now conclude that {an} and {α1r1}n _{+ α}

2r2n } are both solutions

of the recurrence relation a_n = c₁ a_n-1 + c₂ a_{n -2} and both satisfy the initial conditions when n = 0 and n = 1. Because there is a unique

solution of a linear homogeneous recurrence relation of degree two with two initial conditions, it follows that the two solutions are the same, that is, a_n = α₁r₁n _{+ α}

2r2n for all nonnegative integers n. We have

completed the proof by showing that a solution of the linear homogeneous recurrence relation with constant coefficients of degree two must be of the form a_n = α₁r₁n _{+ α}

Second Order Linear Homogeneous

Recurrences: Example A (1)

• Find a solution to

a_n = 5a_n-1 - 6a_n-2 with initial conditions a₀=1, a₁=4

• _{The characteristic equation is}

r2 _{- 5r + 6 = 0}

• The roots are r₁=2, r₂=3

r2 _{- 5r + 6 = (r-2)(r-3)}

• _{Using the 2}nd _{order theorem we have a solution} a_n = ₁2n ₊ 

Second Order Linear Homogeneous Recurrences:

Example A (2)

• _{Given the solution}

a_n = ₁2n ₊ 

23n

• _{We plug in the two initial conditions to get a system of linear equations}

a₀ = ₁20 ₊  230

a₁ = ₁21 ₊  231

• _Thus:

Second Order Linear Homogeneous

Recurrences: Example A (3)

1 = ₁+ ₂ 4 = 2₁+ 3₂

• _{Solving for} ₁ = (1 - ₂), we get

4 = 2₁+ 3₂ 4 = 2(1-₂) + 3₂ 4 = 2 - 2₂ + 3₂

2 = ₂

• _{Substituting for 1: 1 = -1}

• _{Putting it back together, we have}

a_n = ₁2n ₊  23n

Second Order Linear Homogeneous

Recurrences: Example B (1)

• Solve the recurrence

a_n = -2a_n-1 + 15a_n-2

with initial conditions a₀= 0, a₁= 1

• _{If we did it right, we have}

a_n = 1/8 (3)n

- 1/8 (-5)n

Single Root Case

• _{We can apply the theorem if the roots are distincts, i.e. r1r2}

• _{If the roots are not distinct (r1=r2), we say that one characteristic root has}

multiplicity two. In this case, we apply a different theorem

• _{Theorem 2:}

Let c₁, c₂R and suppose that r2 _{- c}

1r - c2 = 0 has only one distinct root, r0, then {an}

is a solution to a_n = c₁a_n-1+ c₂a_n-2 if and only if

a_n= ₁r₀n ₊ 

2nr0n

Single Root Case: Example (1)

• _{What is the solution to the recurrence relation}

a_n = 8a_n-1 - 16a_n-2 with initial conditions a₀= 1, a₁= 7?

• _{The characteristic equation is:}

r2 _{– 8r + 16 = 0}

• _{Factoring gives us:}

r2 _{– 8r + 16 = (r-4)(r-4), so r}

0=4

• _{Applying the theorem we have the solution:}

a_n= ₁(4)n ₊ 

Single Root Case: Example (2)

• _{Given: an= 1(4)}n ₊ 

2n(4)n

•

Using the initial conditions, we get:

a

= 1 =



(4)

+



2

0(4)

=



a

= 7 =



(4)

+



1(4)

= 4



1

+ 4



•

Thus:

=



= 1,



= 3/4

•

The solution is

a

= (4)

+ ¾ n (4)

General Linear Homogeneous

Recurrences

• _{There is a straightforward generalization of these cases to higher-order linear}

homogeneous recurrences

• _{Essentially, we simply define higher degree polynomials} • _{The roots of these polynomials lead to a general solution}

• _{The general solution contains coefficients that depend only on the initial}

conditions

General Linear Homogeneous Recurrences:

Distinct Roots

• _{Theorem 3:}

Let c₁,c₂,..,c_k R and suppose that the characteristic equation rk_{- c}

1rk-1- c2rk-2- … - ck = 0

has k distinct roots r₁,r₂, …,r_k. Then a sequence {a_n} is a solution of the recurrence relation a_n = c₁a_n-1 + c₂a_n-2 + … + c_ka_n-k

if and only if

a_n = ₁r₁n₊ 

2r2n+ … + krkn

General Linear Homogeneous Recurrences:

Any Multiplicity

• Theorem 4:

Let c₁,c₂,..,c_k R and suppose that the characteristic equation rk_{- c}

1rk-1- c2rk-2- … - ck = 0

has t roots with multiplicities m₁,m₂, …,m_t. Then a sequence {a_n} is a solution of the recurrence relation

a_n = c₁a_n-1 + c₂a_n-2 + … + c_ka_n-k if and only if a_n = (_1,0 + _1,1n + … + _1,m1-1nm1-1_)r

1n+

(_2,0 + _2,1n + … + _2,m2-1nm2-1_)r

2n+ ...

(_t,0 + _t,1n + … + _t,mt-1nmt-1_)r tn

Fall 2002 CMSC 203 - Discrete Structures 40

Modeling with Recurrence

Relations

•_Example:

•_{Someone deposits $10,000 in a savings account at a}

bank yielding 5% per year with interest compounded

annually. How much money will be in the account after 30 years?

•_Solution:

Fall 2002 CMSC 203 - Discrete Structures 41

Modeling with Recurrence

Relations

•_{We can derive the following recurrence relation:} •_{Pn = Pn-1 + 0.05Pn-1 = 1.05Pn-1.}

•_{The initial condition is P0 = 10,000.} •_{Then we have:}

•_{P1 = 1.05P0}

•_{P2 = 1.05P1 = (1.05)}2_P 0

•_{P3 = 1.05P2 = (1.05)}3_P 0

•_…

•_{Pn = 1.05Pn-1 = (1.05)}n_P 0

•_{We now have a formula to calculate Pn for any natural number n}

Fall 2002 CMSC 203 - Discrete Structures 42

Modeling with Recurrence

Relations

•_{Let us use this formula to find P30 under the} •_{initial condition P0 = 10,000:}

•_{P30 = (1.05)}30_{10,000 = 43,219.42}

•

Fall 2002 CMSC 203 - Discrete Structures 43

Solving Recurrence Relations

•_{In general, we would prefer to have an explicit formula}

to compute the value of a_n rather than conducting n iterations.

•_{For one class of recurrence relations, we can obtain such}

formulas in a systematic way.

•_{Those are the recurrence relations that express the}

Fall 2002 CMSC 203 - Discrete Structures 44

Solving Recurrence Relations

•_{Definition: A linear homogeneous recurrence relation of}

degree k with constant coefficients is a recurrence relation of the form:

•_{an = c1an-1 + c2an-2 + … + ckan-k,}

•_{Where c1, c2, …, ck are real numbers, and ck  0.}

•_{A sequence satisfying such a recurrence relation is}

uniquely determined by the recurrence relation and the k initial conditions

Fall 2002 CMSC 203 - Discrete Structures 45

Solving Recurrence Relations

•_Examples:

•_{The recurrence relation Pn = (1.05)Pn-1 is a linear}

homogeneous recurrence relation of degree one.

•_{The recurrence relation fn = fn-1 + fn-2 is a linear}

homogeneous recurrence relation of degree two.

•_{The recurrence relation an = an-5 is a linear homogeneous}

Fall 2002 CMSC 203 - Discrete Structures 46

Solving Recurrence Relations

•_{Basically, when solving such recurrence relations, we try}

to find solutions of the form a_n = rn_{, where r is a constant.}

•_{an = r}n _{is a solution of the recurrence relation} a_n = c₁a_n-1 + c₂a_n-2 + … + c_ka_n-k if and only if

•_rn _{= c}

1rn-1 + c2rn-2 + … + ckrn-k.

•_{Divide this equation by r}n-k _{and subtract the right-hand} side from the left:

•_rk _{- c}

1rk-1 - c2rk-2 - … - ck-1r - ck = 0

•_{This is called the characteristic equation of the}

Fall 2002 CMSC 203 - Discrete Structures 47

Solving Recurrence Relations

•_{The solutions of this equation are called the characteristic}

roots of the recurrence relation.

•_{Let us consider linear homogeneous recurrence relations of}

degree two.

•_{Theorem: Let c1 and c2 be real numbers. Suppose that r}2 _– c₁r – c₂ = 0 has two distinct roots r₁ and r₂.

•_{Then the sequence {an} is a solution of the recurrence}

relation a_n = c₁a_n-1 + c₂a_n-2 if and only if a_n = ₁r₁n ₊ 

Fall 2002 CMSC 203 - Discrete Structures 48

Solving Recurrence Relations

•_{Example: What is the solution of the recurrence relation}

a_n = a_n-1 + 2a_n-2 with a₀ = 2 and a₁ = 7 ?

•_{Solution: The characteristic equation of the recurrence}

relation is r2 _{– r – 2 = 0.}

•_{Its roots are r = 2 and r = -1.}

•_{Hence, the sequence {an} is a solution to the recurrence}

relation if and only if:

•_{an = 12}n ₊ 

Fall 2002 CMSC 203 - Discrete Structures 49

Solving Recurrence Relations

•_{Given the equation an = 12}n ₊ 

2(-1)n and the initial conditions a₀ = 2 and a₁ = 7, it follows that

•_{a0 = 2 = 1 + 2}

•_{a1 = 7 = 12 + 2 (-1)}

•_{Solving these two equations gives us}

₁ = 3 and ₂ = -1.

•_{Therefore, the solution to the recurrence relation and}

initial conditions is the sequence {a_n} with

Fall 2002 CMSC 203 - Discrete Structures 50

Solving Recurrence Relations

•_{Example: Give an explicit formula for the Fibonacci}

numbers.

•_{Solution: The Fibonacci numbers satisfy the recurrence}

relation f_n = f_n-1 + f_n-2 with initial conditions f₀ = 0 and f₁ = 1.

•_{The characteristic equation is r}2 _{– r – 1 = 0.}

•_{Its roots are}

Fall 2002 CMSC 203 - Discrete Structures 51

Solving Recurrence Relations

•_{Therefore, the Fibonacci numbers are given by}

for some constants _{1 and} _2.

We can determine values for these constants so that the sequence meets the conditions f₀ = 0 and f₁ = 1:

n n n f _                2 5 1 2 5 1 2 1   0 2 1

0     f 1 2 5 1 2 5 1 2 1

1  

Fall 2002 CMSC 203 - Discrete Structures 52

Solving Recurrence Relations

•_{The unique solution to this system of two equations and}

two variables is

So finally we obtained an explicit formula for the Fibonacci numbers: 5 1 , 5 1 2

1    

Fall 2002 CMSC 203 - Discrete Structures 53

Solving Recurrence Relations

•_{But what happens if the characteristic equation has only}

one root?

•_{How can we then match our equation with the initial}

conditions a₀ and a₁ ?

•_{Theorem: Let c1 and c2 be real numbers with c2 0.}

Suppose that r2 _{– c}

1r – c2 = 0 has only one root r0.

A sequence {a_n} is a solution of the recurrence relation a_n = c₁a_n-1 + c₂a_n-2 if and only if

a_n = ₁r₀n ₊ 

Fall 2002 CMSC 203 - Discrete Structures 54

Solving Recurrence Relations

•_{Example: What is the solution of the recurrence relation an =}

6a_n-1 – 9a_n-2 with a₀ = 1 and a₁ = 6?

•_{Solution: The only root of r}2 _{– 6r + 9 = 0 is r}

0 = 3.

Hence, the solution to the recurrence relation is a_n = ₁3n ₊ 

2n3n

for some constants ₁ and ₂. To match the initial condition, we need

a₀ = 1 = ₁

a₁ = 6 = ₁3 + ₂3

•_{Solving these equations yields 1 = 1 and 2 = 1.} •Consequently, the overall solution is given by

Compound interest

• Given

• P = initial amount (principal) • n = number of years

• r = annual interest rate

• A = amount of money at the end of n years At the end of:

1 year: A = P + rP = P(1+r)

2 years: A = P + rP(1+r) = P(1+r)2

3 years: A = P + rP(1+r)2 = P(1+r)3 …

Eugene Catalan

• Belgian mathematician, 1814-1894

• Catalan numbers are generated by the formula:

C_n = C(2n,n) / (n+1) for n > 0

• The first few Catalan numbers are:

n 0 1 2 3 4 5 6 7 8 9 10 11

Catalan Numbers: applications

The number of ways in which a polygon with n+2 sides

can be cut into n triangles

The number of ways in which parentheses can be

placed in a sequence of numbers, to be multiplied two at a time

• The number of rooted trivalent trees with n+1 nodes

• The number of paths of length 2n through an n by n grid that do not rise above the main diagonal

Towers of Hanoi

Start with three pegs numbered 1, 2 and 3

mounted on a board, n disks of different sizes with holes in their centers, placed in order of increasing size from top to bottom.

Rules of the game: Hanoi towers

Start with all disks stacked in peg 1 with the

smallest at the top and the largest at the bottom • Use peg number 2 for intermediate steps

End of game: Hanoi towers

• Game ends when all disks are stacked in peg number 3 in the same order they were stored at the start in peg number 1.

• Verify that the minimum number of moves needed is the

Catalan number C₃ = 5.

How to Cheat with Maple (1)

• _{Maple and other math tools are great resources. However, they are no}

substitutes for knowing how to solve recurrences yourself

• _{As such, you should only use Maple to check you answers}

• _{Recurrence relations can be solved using the rsolve command and giving}

Maple the proper parameters

• _{The arguments are essentially a comma-delimited list of equations}

• General and boundary conditions

How to Cheat with Maple (2)

> rsolve({T(n)= T(n-1)+2*n,T(1)=5},T(n)); 1+2(n+1)(1/2n+1)-2n

• You can clean up Maple’s answer a bit by encapsulating it in the simplify command