Recurrence and Recursion - PowerPoint Presentation

Slide1

Recurrence and Recursion

Introduced through Computer ScienceSlide2

Recursively Defined Sequences

Three ways to define a sequence:

Informal: write the first few terms, expecting the pattern to be obvious

Traditional: write a formula to describe the sequenceComputer Science: use a recurrence relationshipWrite a formula to relate each future term to a number of past termsSpecify initial conditions, the first few values needed to satisfy the formulaEX: is the recurrence relationship and are the initial conditions required to calculate the sequenceSlide3

Why Use Recursion?

Sometimes, you can’t define a sequence with a formula, but you usually can define it with recursion

Defining a sequence with recursion is equivalent to proving the sequence with induction

Recursion is a fundamental programming technique because it breaks large tasks into their smallest possible chunksThis translates to highly efficient code, if done correctlySlide4

Computing Recursive Terms

Recurrence relation: for all integers k ≥ 2

, initial conditions:

Find C[2], C[3], C[4]C[2]=C[1]+2*C[0]+ 1C[2] = 2 + 2*1 + 1 = 5C[3] = C[2] + 3*C[1] + 1C[3] = 5 + 3*2 + 1 = 12C[4] = C[3] + 4*C[2] + 1C[4] = 12 + 4*5 + 1 =

33Slide5

Re-Writing a Recurrence Relationship

Just like sums and products, variables in a recurrence formula can be expressed in multiple ways

For all integers k ≥ 1,

For all integers k ≥ 0,Both of these relationships say the same thingNote that the recurrence relationship does not define any specific sequence. Rather, it describes the patterns to expect in a sequence with given initial conditions Also, a recursive sequence doesn’t have to start at element 0. Initial conditions may be placed at any index valueSlide6

Showing an Explicit Formula Satisfies A Recursive Formula

Show that the sequence for n ≥ 0 satisfies the relationship for all k ≥ 1

Based on the formula, S[0] = 1

S[k] and S[k-1] are defined in the recursion, so substitute both into the explicit formulaS[k] = (-1)^k * k!S[k-1] = (-1)^(k-1) * (k-1)!(-k)*S[k-1] = (-k)*{(-1)^(k-1)*(k-1)!} – By substitution from recurrence formula(-1)(-1)^(k-1) * k(k-1)! – By simplification(-1)^(k) * K! – By basic algebra (EX: law of exponents)

Final: (-k)*S[k-1] = S[k] – By substitution from explicit formulaSlide7

Fibonacci Numbers

A single pair of rabbits is born at the beginning of the year. Suppose the following:

Rabbit pairs are infertile for one month after birth, but then give birth to one new male/female pair each month

No rabbits dieHow many rabbits will there be at the end of the year?Try to solve this yourself first. Then check out the programSlide8

Solution

For every integer

n

≥ 1, let F[n] = [number of rabbit pairs alive at end of month n]F[0] = the initial number of pairs = 1Note that the number of pairs born in month n equals the number of pairs alive in month n-2.

F[k] = F[k-1] + F[k-2]F[1] is also 1 because the first pair is not fertile until the second monthCalculation:F[2] = F[1] + F[0] = 1 + 1 = 2F[3] = F[2] + F[1] = 2 + 1 = 3F[4] = F[3] + F[2] = 3 + 2 = 5

F[5] = F[4] + F[3] = 5 + 3 = 8….F[12] = F[11] + F[10] = 144 + 89 = 233This is the number of pairs. Double it to get 466 rabbits in allSlide9

Solving Recurrence Relationships by Iteration

Given a sequence that satisfies a recurrence relationship with initial conditions, there may exist a formula that can represent that sequence

This is sometimes very useful

EX: if you have terms 1-10 of the sequence, but you need terms 10,000-100,000, you would have to iterate the recurrence relation many timesWith a formula, known as a solution, you could simply plug in a value without having to iterate the relation to obtain a resultSlide10

Method of Iteration

This is the most simple method

Start with the relation and the initial conditions and iterate it until you see a pattern in the sequence produced. Based on the pattern, guess the formula

EX: for the relation “a[k] = a[k-1] + 2” and the condition a[0] = 1, find the formulaStart with a few terms and substitute numerical expressions into each term instead of numbersa[1] = a[0] + 2 = 1 + 2a[2] = a[1] + 2 = (1 + 2) + 2 = 1 + 2 + 2 – eliminate parenthesis

a[3] = a[2] + 2 = (1 + 2 + 2) +2 = 1+2+2+2Slide11

Iteration Continued

By rewriting 1 + 2 + 2 + 2 as 1 + 3 * 2, and by noting that 3 is the index of a[3], a pattern can be observed

It appears that the recurrence relation with a[0] = 1 is equivalent to the formula a[n] = 1 + 2n

Note: this is only a guess. To be absolutely certain, you must prove the equivalence of the two formulaeSlide12

Arithmetic Sequences

A sequence “a[0], a[1], a[2]…” is an arithmetic sequence

iff

there is a constant, d, such that:a[k] = a[k-1] + d for all integers K ≥ 1Or equivalently: a[n] = a[0] + d*n for all integers n ≥ 0In an arithmetic sequence, each term equals the previous term plus a fixed constantSlide13

Geometric Sequences

A sequence is geometric

iff

there is a constant, r, such that:a[k] = r * a[k-1] for all integers k ≥ 1Or a[n] = a[0] * r^n for all integers n ≥ 0In a geometric sequence, each term equals the previous term times a fixed constantUnlike an arithmetic sequence, which is linear, a geometric sequence is exponential (the formula contains a variable as an exponent), which implies that it will change very rapidlySlide14

Using Formulas to Simplify Solutions

The Tower of Hanoi relation is M[k] = 2*m[k-1] +1 for all integers k ≥ 2. m[1] = 1

By iteration, we find that:

m[2] = 2m[1] + 1 = 2*1 + 1 = 2^1 + 1m[3] = 2m[2] + 1 = 2(2 + 1) + 1 = 2^2 + 2 + 1m[4] = 2m[3] + 1 = 2(2^2 + 2 + 1) = 2^3 + 2^2 + 2 + 1It seems that each term, m[k] equals the sum of successive powers of two, (2^(k-1)+…+2^2+2^1+2^0)By the formula for the sum of a geometric sequence, we find that the sum equals

The same process can be used to sum an arithmetic sequence using it’s respective formulaSlide15

Checking Correctness with Induction

It is possible to make mistaken assumptions when solving for a recurrence relation, so its correctness must be verified

First, show the formula holds to the relation for the first term a[1]

Second, show that for all integers k ≥ 1, if the formula holds for n = k in a[n], then it holds for n = k+1The format of a proof by induction and the procedures by which the terms to be proven equivalent are compared with one another are detailed in an accompanying moduleSlide16

Discovering that an Explicit Formula is Incorrect

Given c[k] = 2c[k-1] + k for all integers k ≥ 1 with c[0] = 1

Is the formula “c[n] = 2^n + n for all integers n ≥ 0” correct?

Basis step: c[0] = 1 and 2^0 + 1 = 1Inductive step: c[k] = 2^k + k for some integer k ≥ 0We must show: c[k+1] = 2^(k+1) + (k+1) c[k+1] = 2c[k] + (K + 1) – By the recurrence relationshipc[k+1] =2(2^k + k) + (k+1) – We assume c[k] = 2^k + k c[k+1] = 2^(k+1) + 3k + 1 – Basic algebraSlide17

…Continued

Now we must show holds true

However, this simplifies algebraically to 2*k = 0 meaning that k = 0

The reason this makes the relation false is deceptively simple. k is meant to be any non-negative integer. The fact that it only equals zero means that it cannot satisfy the sequence defined by the recurrence relationshipSlide18

An Interesting Case

If you would like to see some more complicated techniques, one of which will let you solve the Fibonacci sequence, check

this

outSlide19

Other Aspects of Recursion

Recursive thinking can be applied to sets of any kind

There are three formal requirements to apply recursion

Base: A statement that certain objects belong to a setRecursion: A collection of rules indicating how to form new set objects from those already in the setRestriction: A statement that no other objects are in the set except for those already definedSlide20

A Simple Example

There are certain combinations of parenthesis that are illegal, like ()))

To build a set of parenthesis that contains only legal combinations, use the following recursive definition

Base: () is in set PRecursion:If E is in P, so is (E)If E and

F are in P, so is EFRestriction: Only combinations derived from the given rules are in PBy this definition, the combinations (), ()(), (())(), (()()), and others can be formed, so the set is infinite

This method is recursive because you start with the initial condition () and use a well-defined procedure to obtain the setSlide21

Proving Properties of Recursive Sets

A process called structural induction is used to prove recursively defined sets

Let S be a recursively defined set with some property that is to be proven for every element of the set. The following must be done:

1. Show that each object in the base satisfies the property2. Show that for each rule in recursion, if the rule is applied to an object that satisfies the property, then the object defined by the rule also satisfies the propertyBecause only the objects defined by the base and the rules are in S, all of the objects in S must satisfy the propertySlide22

Proving the Set of Legal Parenthesis

Let the property of the set be “The parenthesis configuration has an equal number of left and right parenthesis”

The Base of P is () which has 1 left and 1 right parenthesis

Proof of RecursionIf E is in the set, it satisfies the property, so (E) also must satisfy the property because 1 left and 1 right parenthesis are addedIf

E and F are in the set, then they have n and m pairs of parenthesis, respectively. Therefore,

EF, which is simply E and F placed next to each other, must have n

+ m pairsTherefore, all of the elements in P satisfy the propertySlide23

Recursive Sums and Products

The recursive forms of sums and products are designed to specify the order in which they are to be computed

EX:

If you began with a value of 4 for n, and you continued to recursively expand this product form, you would end with ((a[1]*a[2])*a[3])*a[4] which is a precise definition of the order by which the product is calculatedThis form applies to both sums and products and is the method of choice when using algorithms to compute them. It is also used with induction to establish properties of finite setsThe following code is used to recursively sum a set

Sum

:=0

For k:= 1 to

n

sum

:=

sum

+ a[k]

Next

kSlide24

Recursive Unions and Intersections

Without diving into the definition of unions and intersections, they can be expanded recursively in the same way sums and products can be

EX:

This definition can also be used to prove properties with induction and applies to both unions and intersectionsAs with recursive sums and products, this form specifies the order by which unions and intersections are to be performed, which is necessary to compute them using an algorithm Slide25

Recursive Functions

A function is recursive if it refers to itself in its definition

Consider McCarthy’s 91 function: for all positive integers

nRemarkably, all values less than 101 yield 91 after successive iteration of the functionFor values above 101, the function is still well defined because n-10 will be calculatedAnother well-defined function is the Ackermann function

, which is famous for its incredible growth rateSlide26

Poorly Defined Recursive Functions

It is often difficult to define a function recursively. Consider the following for n ≥ 1

Why is this undefined?G(1) = 1G(2) = 1 + G(1) = 2G(3) = G(8) = 1 + G(4) =…=1+(1+2) = 4G(5) = G(14) = 1+G(7) =…= 3+G(5) As you can see, at value 5, you get G(5) = 3 + G(5), so 0 = 3. This means that the expression can’t be logically defined as a function

This is clearly undefined, but some recursive functions are not easily discerned as functions or notSlide27

Why Use Recursion?

Computers can do a lot of nifty things, but they are not mathematicians (yet!). Computers cannot look at the sum of a set or casually scan the definition of a function and simply “jump to” an answer. Computers operate on a step-by-step basis. Recursion is one of the most efficient ways to implement a step-by-step process. One could easily write thousands of lines of code to sum a set or calculate a sequence. By using recursion correctly, you can replace those lines with a compact, re-useable module