2016W2 Introduction to Induction Steve Wolfman 1 Outline Prereqs Learning Goals and Quiz Notes Bonus Prelude A Pattern for Induction Problems and Discussion SingleElimination Tournaments ID: 560403
Download Presentation The PPT/PDF document "CPSC 121: Models of Computation" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
CPSC 121: Models of Computation2016W2
Introduction to InductionSteve Wolfman
1Slide2
OutlinePrereqs, Learning Goals, and Quiz Notes
Bonus PreludeA Pattern for InductionProblems and DiscussionSingle-Elimination Tournaments
Induction on Numbers
Next Lecture Notes
2Slide3
Quiz Notes: Summations
Let’s play with these:
3
Slide4
Quiz Notes
You’re faced with a problem like “How many teams can play in a 100 round tournament?” or “How many teams can play in an n-round tournament?”You don’t know the answer.What can you do?
Strategies:
Try to clearly define terms (What’s a tournament? What’s a round?)
Try a
smaller number. 100 is hard, what about 1 or 2?Write the problem (or a small version) out as a story and figure out which parts of the story matter.
4Slide5
Learning Goals: Pre-Class
By the start of class, you should be able to:Convert sequences to and from explicit formulas that describe the sequence.
Convert sums to and from summation/“sigma” notation.
Convert products to and from product/“pi” notation.
Manipulate formulas in summation/product notation by adjusting their bounds, merging or splitting summations/products, and factoring out values.
5Slide6
Learning Goals: In-Class
By the end of this unit, you should be able to:Given a theorem to prove via induction and the insight into how the problem breaks down into one or more smaller problem(s), write out a complete proof strategy.
P
rove naturally self-referential properties by induction. (That is, identify the “insight” and flesh out the strategy into a complete proof.)
Discuss point of learning goals.
6Slide7
Where We Are inThe Big Stories
TheoryHow do we model computational systems?
Now
: Developing a new proof technique that’s perfect for algorithms involving recursion (or iteration)... which is almost all interesting algorithms!
Hardware
How do we build devices to compute? Now: Taking a break in lecture. In lab, nearly at complete, working computer! (Although lab’s taking a break too, for regular expressions.)
7Slide8
OutlinePrereqs, Learning Goals, and Quiz Notes
Bonus PreludeA Pattern for InductionProblems and DiscussionSingle-Elimination Tournaments
Induction on Numbers
Next Lecture Notes
8Slide9
Prelude: A Binary Tree with n Nodes contains n
+1 Empty TreesNote: We’ll define a binary tree as one of an empty tree or a “node” with two subtrees (each itself a binary tree).
(What a “node” is depends on what we want to do with the tree but doesn’t concern us here!)
Translate the theorem to predicate logic; use the proofs sheet to think about how to prove it.
9Slide10
Induction: Regular Old ProofPlus One Powerful Step
A binary tree is: an empty tree or a node with two subtrees (each itself a binary tree).
Theorem
: A binary tree with
n
nodes contains n+1 empty trees.
10
Induction gives us one extra tool:
assuming our theorem holds for recursive sub-cases.Slide11
“Binary Trees”
A “binary tree” is either an empty tree or a node with two children, each of which is a binary tree.
11
(They let us do some cool CS things... see CS110, 210, 221
.)
This one happens to be a binary
search
tree.
20
9
2
15
5
10
17
7Slide12
New Proof Technique: (Structural) Induction
Identify the key recursive structure in the problem.For example, binary trees.
A binary tree is one of:
An empty tree
A node with two
subtrees, each of which is a binary tree
12
The book discusses “weak” and “strong” induction.
As far as we are concerned, there can be only one induction: structural.Slide13
New Proof Technique: (Structural) Induction
Circle each recursive appearance of the structure inside its definition.
A binary tree is one of:
An empty tree
A node with two subtrees, each of which is a binary tree
13
So, the two subtrees are recursive appearances of binary trees.Slide14
New Proof Technique: (Structural) Induction
Divide the cases into: those without recursive appearances (“base cases”) and those with (“recursive” or “inductive” cases)
A binary tree is one of:
An empty tree
A node with two subtrees, each of which is a binary tree
14
Base case
Inductive case
Reuse
these first three steps whenever
you reuse the recursive structure.Slide15
New Proof Technique: (Structural) Induction
Write a predicate describing what you want to say about the structure.
For example:
P(
t
) ≡ the number of empty trees in the binary tree t is one more than the number of nodes in t
15
Always in terms of one of the recursive structure
.
Be careful… it’s easy to make this not a predicate:
It should be true or false when you plug in a parameter (not, e.g., a number)!Slide16
New Proof Technique: (Structural) Induction
16
This step is easy!
Slide17
New Proof Technique: (Structural) Induction
Write out your proof template:P(
a
) ≡
[fill in predicate definition]
Theorem: For all [recursive structure] a, P(a) holds.Proof by structural induction:Base case: [For each base case you ID’d, write the structure of that case and prove the theorem holds for that case. If you don’t END by showing your theorem true for the base case structure, you probably messed up!]
17Slide18
Example
P(t) ≡ the number of empty trees in the binary tree
t
is one more than the number of nodes in
t
Theorem: For all binary trees t, P(t) holds.Proof by structural induction:Base case: t is an empty tree.In that case, t has no nodes and one empty tree.1 = 0 + 1 as expected.
18Slide19
New Proof Technique: (Structural) Induction
Proof template continued:Inductive Step:
[For each recursive case write out an inductive step…]
Consider an arbitrary
[recursive case structure] a.Induction Hypothesis: Assume the theorem holds for [each recursive appearance of the structure in this case].We now show that the theorem holds for a itself.
19Slide20
Example
Inductive Step:Consider an arbitrary non-empty binary tree
t
(
i.e., a node with two subtrees, each of which is a binary tree).Induction Hypothesis: Assume the theorem holds for the two subtrees of t.We now show that the theorem holds for t itself.
20Slide21
New Proof Technique: (Structural) Induction
Proof template completed:[Show that theorem holds! Three signs that something has probably gone wrong:
You don’t
END
by showing that your theorem holds for
a.You don’t USE the “Induction Hypothesis” assumption(s) you made.You EVER take a recursive appearance and use the recursive definition to break it down further (Example BAD thing to say: “Each of the subtrees is also a binary tree; so, it can have subtrees and so on until we reach an empty tree.”)]
21Slide22
Example
Consider an arbitrary non-empty binary tree t
.
Induction Hypothesis:
Assume the theorem holds for
the two subtrees of t.t’s left subtree tl has some number of nodes nl and some number of empty trees el. By assumption, el = nl + 1. Similarly for t’s right subtree tr, er = nr + 1.t’s number of nodes n
includes all of tl’s and tr’s nodes plus its own node. t’s number of empty trees e includes all and only the empty trees in tl
and
t
r
.
Thus
n
=
n
l
+
n
r
+ 1 and
e
= el + er. Substituting from above: e = el
+ er = (nl + 1) + (nr + 1) = (n
l
+
n
r
+ 1) + 1 =
n + 1.Thus, e = n + 1, as we wanted to prove.QED
22Slide23
OutlinePrereqs, Learning Goals, and Quiz Notes
Bonus PreludeA Pattern for InductionProblems and Discussion
Single-Elimination Tournaments
Induction on Numbers
Next Lecture Notes
23Slide24
A Pattern for Induction
Identify the recursive structure in the problem.Circle each recursive appearance of the structure inside its definition.
Divide the cases into: those without recursive appearances (“base cases”) and those with (“recursive” or “inductive” cases)
Write a predicate P(
a
) describing what you want to say about the structure.Write your theorem a?,P(a)Complete the proof template.
24Slide25
25
P(
a
) ≡
[fill in predicate definition]
Theorem:
For all
[recursive structure]
a
, P(
a
) holds.
Proof by structural induction:
Base case:
[For each non-recursive case, prove the theorem holds for that case. End by showing your theorem true for the base case structure!]
Inductive Step:
[For each recursive case…]
Consider an arbitrary
[recursive case structure]
a
.
Induction Hypothesis:
Assume the theorem holds for
[each recursive appearance of the structure in this case]
.
We now show the theorem holds for a. [And then do so! You should:END by showing that your theorem holds for a.USE the “Induction Hypothesis” assumption(s) you made.
NEVER take a recursive appearance and use the recursive definition to break it down further, like this BAD example: “each subtree is a binary tree that can have subtrees and so on until we reach an empty tree.”]
It helps to write out what you want to prove rather than just “show the theorem holds for
a
”.
(Even though neither one is strictly necessary.)Slide26
OutlinePrereqs, Learning Goals, and Quiz Notes
Bonus PreludeA Pattern for InductionProblems and Discussion
Single-Elimination Tournaments
Induction on Numbers
Next Lecture Notes
26Slide27
Single-Elimination Tournaments
In each round teams play in pairs. Losing teams are eliminated. The tournament ends when only one team remains.
27Slide28
What is a Tournament?
Let’s think like CPSC 110ers…A tournament is one of:
28Slide29
The “insight into how the problem breaks down
”
A tournament of
n
rounds is made up of two tournaments of (
n-1) rounds, like how the Stanley Cup Finals is set “on top of” the “east half” and “west half” tournaments.
29
One extra round:
east winner vs. west winnerSlide30
Single-Elimination Tournament Problem
What’s the maximum number of teams in a 0
-round single-elimination tournament?
0 teams
1 team
2 teams3 teamsNone of these
30
Think: what does this correspond to in the data definition?Slide31
Single-Elimination Tournament Problem
What’s the maximum number of teams in a 1-round single-elimination tournament?
0 teams
1 team
2 teams
3 teamsNone of these
31
A “one round tournament” has only one set of games.
A “two round tournament” has two sets of games (finals and semi-finals).Slide32
Single-Elimination Tournament Problem
What’s the maximum number of teams in a 2-round single-elimination tournament?
0 teams
1 team
2 teams
3 teamsNone of these
32
Would there be any “byes” in the tournament?
A team with a “bye” gets to advance to the next round “for free”.Slide33
Single-Elimination Tournament Problem
What’s the maximum number of teams in an n
-round single-elimination tournament?
n
teams2n teamsn2 teams2n teamsNone of these
33
(Induction is for proving our theorem;
it’s
not
for figuring out what we want to prove!)Slide34
Side Note: How Many Rounds Does a Tournament Have?
How many rounds does a tournament that’s just a single winner have?
How many rounds does a tournament that’s a final game between the winners of two smaller tournaments have?
34Slide35
Side Note: How Many Rounds Does a Tournament Have?
A tournament has:
0 rounds if it’s just a single “winner”
1 more round than the largest number of rounds of either of its
subtournaments
, otherwise
35Slide36
36
P(
a
) ≡
[fill in predicate definition]
Theorem:
For all
[recursive structure]
a
, P(
a
) holds.
Proof by structural induction:
Base case:
[For each non-recursive case, prove the theorem holds for that case. End by showing your theorem true for the base case structure!]
Inductive Step:
[For each recursive case…]
Consider an arbitrary
[recursive case structure]
a
.
Induction Hypothesis:
Assume the theorem holds for
[each recursive appearance of the structure in this case]
.
We now show the theorem holds for a. [And then do so! You should:END by showing that your theorem holds for a.USE the “Induction Hypothesis” assumption(s) you made.
NEVER take a recursive appearance and use the recursive definition to break it down further, like this BAD example: “each subtree is a binary tree that can have subtrees
and so on until we reach an empty tree.”]Slide37
37
P(
a
) ≡
there are at most 2
n
teams in tournament t with n rounds
Theorem:
For all
[recursive structure]
a
, P(
a
) holds.
Proof by structural induction:
Base case:
[For each non-recursive case, prove the theorem holds for that case. End by showing your theorem true for the base case structure!]
Inductive Step:
[For each recursive case…]
Consider an arbitrary
[recursive case structure]
a
.
Induction Hypothesis:
Assume the theorem holds for [each recursive appearance of the structure in this case].We now show the theorem holds for
a. [And then do so! You should:END by showing that your theorem holds for a.USE
the “Induction Hypothesis” assumption(s) you made.
NEVER
take a recursive appearance and use the recursive definition to break it down further, like this
BAD
example: “each
subtree
is a binary tree that can have
subtrees
and so on until we reach an empty tree.”]Slide38
38
P(
a
) ≡
there are at most 2
n
teams in tournament t with n rounds
Theorem:
For all
tournaments
t
, P(
t
) holds.
Proof by structural induction:
Base case:
[For each non-recursive case, prove the theorem holds for that case. End by showing your theorem true for the base case structure!]
Inductive Step:
[For each recursive case…]
Consider an arbitrary
[recursive case structure]
t
.
Induction Hypothesis:
Assume the theorem holds for
[each recursive appearance of the structure in this case].We now show the theorem holds for t.
[And then do so! You should:END by showing that your theorem holds for t.USE the “Induction Hypothesis” assumption(s) you made.
NEVER
take a recursive appearance and use the recursive definition to break it down further, like this
BAD
example: “each
subtree
is a binary tree that can have
subtrees
and so on until we reach an empty tree.”]
I changed all the variables to t just because t’s a good name for a
t
ournament.Slide39
39
P(
a
) ≡
there are at most 2
n
teams in tournament t with n rounds
Theorem:
For all
tournaments
t
, P(
t
) holds.
Proof by structural induction:
Base case:
A tournament that is just a single “winner”…
[TODO: finish!]
Inductive Step:
[For each recursive case…]
Consider an arbitrary
[recursive case structure]
t
.
Induction Hypothesis: Assume the theorem holds for [each recursive appearance of the structure in this case].We now show the theorem holds for t. [And then do so! You should:END by showing that your theorem holds for t.
USE the “Induction Hypothesis” assumption(s) you made.NEVER take a recursive appearance and use the recursive definition to break it down further, like this BAD example: “each
subtree
is a binary tree that can have
subtrees
and so on until we reach an empty tree.”]
We’ll leave the proof of the base case until we’ve finished the structural pieces.Slide40
40
P(
a
) ≡
there are at most 2
n
teams in tournament t with n rounds
Theorem:
For all
tournaments
t
, P(
t
) holds.
Proof by structural induction:
Base case:
A tournament that is just a single “winner”…
[TODO: finish!]
Inductive Step:
Consider an arbitrary
[recursive case structure]
t
.
Induction Hypothesis:
Assume the theorem holds for
[each recursive appearance of the structure in this case].We now show the theorem holds for t
. [And then do so! You should:END by showing that your theorem holds for t.USE
the “Induction Hypothesis” assumption(s) you made.
NEVER
take a recursive appearance and use the recursive definition to break it down further, like this
BAD
example: “each
subtree
is a binary tree that can have
subtrees
and so on until we reach an empty tree.”]
There’s just the one recursive case.Slide41
41
P(
a
) ≡
there are at most 2
n
teams in tournament t with n rounds
Theorem:
For all
tournaments
t
, P(
t
) holds.
Proof by structural induction:
Base case:
A tournament that is just a single “winner”…
[TODO: finish!]
Inductive Step:
Consider an arbitrary
tournament
t
that is a final game between the winners of two
subtournaments
(which we’ll call s
1
and s2
).Induction Hypothesis: Assume the theorem holds for [each recursive appearance of the structure in this case].We now show the theorem holds for t. [And then do so! You should:
END
by showing that your theorem holds for
t
.
USE
the “Induction Hypothesis” assumption(s) you made.
NEVER
take a recursive appearance and use the recursive definition to break it down further, like this
BAD
example: “each
subtree
is a binary tree that can have
subtrees
and so on until we reach an empty tree.”]
It’s often handy to
name
those recursive appearances!
E.g., subtournaments s
1
and s
2
.Slide42
42
P(
a
) ≡
there are at most 2
n
teams in tournament t with n rounds
Theorem:
For all
tournaments
t
, P(
t
) holds.
Proof by structural induction:
Base case:
A tournament that is just a single “winner”…
[TODO: finish!]
Inductive Step:
Consider an arbitrary
tournament
t
that is a final game between the winners of two
subtournaments
s
1
and s2
.Induction Hypothesis: Assume the theorem holds for s1 and s2.We now show the theorem holds for
t
.
[
And then do so!
You should:
END
by showing that your theorem holds for
t
.
USE
the “Induction Hypothesis” assumption(s) you made.
NEVER
take a recursive appearance and use the recursive definition to break it down further, like this
BAD
example: “each
subtree
is a binary tree that can have
subtrees
and so on until we reach an empty tree.”]
Slide43
43
It helps to write out what you want to prove rather than just “show the theorem holds for
a
”.Slide44
44
Slide45
45
Slide46
46
Slide47
47
Slide48
The “insight into how the problem breaks down
”
The
n
-round tournament with the maximum number of teams is made from two
(n-1)-round tournaments with the maximum number of teams.
48Slide49
Step-by-Step?
Here’s the logical structure of our theorems:MT(0,2
0
)
.
nZ0, (n > 0) MT(n-1,2n-1) MT(n,2n).
Do these prove MT(1,21)?Yes.No.
I don’t know.
49Slide50
Step-by-Step?
Here’s the logical structure of our theorems:MT(0,2
0
)
.
nZ0, (n > 0) MT(n-1,2n-1) MT(n,2n).
Plus, we know MT(1,21). Do all of these prove MT(2,2
2
)
?
Yes.
No.
I don’t know.
50Slide51
Step-by-Step?
Here’s the logical structure of our theorems:MT(0,2
0
)
.
nZ0, (n > 0) MT(n-1,2n-1) MT(n,2n).
Plus, we know MT(1,21) and MT(2,22
)
.
Do all of these prove
MT(3,2
3
)
?
Yes.
No.
I don’t know.
51Slide52
Step-by-Step?
Here’s the logical structure of our theorems:MT(0,2
0
)
.
nZ0, (n > 0) MT(n-1,2n-1) MT(n,2n).
From this, can we prove MT(n,2n)
for any particular integer
n
?
Yes.
No.
I don’t know.
52Slide53
The Magic of Induction!
Our “proof” is really a recursive program.Plug in a tournament of a particular number of rounds n, and it produces a proof that
that
tournament cannot have more than 2
n
teams in it.The magic of induction is that we agree that writing this program is the same as writing all the proofs the program can create.
53Slide54
Outline
Prereqs, Learning Goals, and Quiz NotesBonus Prelude
A Pattern for Induction
Problems and Discussion
Single-Elimination Tournaments
Induction on NumbersNext Lecture Notes
54Slide55
Natural Numbers and Induction
Can we prove things inductively about the natural numbers without a recursive data definition? Are they “self-referential”?Here’s one answer:
A natural number is:
0
The next number after (that is, the “successor” of) a natural number
55Slide56
Natural Numbers and Induction
Can we prove things inductively about the natural numbers without a recursive data definition? Are they “self-referential”?But, let’s just try one!
Problem
: What is the sum of the natural numbers 0..
n
?
56Slide57
Partly Worked Problem: Sum of 0..n
Partly Worked Problem: What is the sum of the natural numbers 0..
n
?
Induction doesn’t help with insight...
But spoilers do: n(n+1)/2.Now, how do we prove it?
57Slide58
Partly Worked Problem: Sum of 0..n
Partly Worked Problem: Prove that the sum of the natural numbers 0..
n
is n(n+1)/2.
Is this self-referential? If we knew the sum of the numbers 0..(
n-1), would it tell us anything about the sum of the numbers 0..n?
58Slide59
Sigma, Pi, Factorial, Powers:All Self-Referential
59
So? So, you can do inductive proofs on them!Slide60
Outline
Prereqs, Learning Goals, and Quiz NotesBonus Prelude
A Pattern for Induction
Problems and Discussion
Single-Elimination Tournaments
Induction on NumbersNext Lecture Notes
60Slide61
Learning Goals: In-Class
By the end of this unit, you should be able to:Establish properties of self-referential structures using inductive proofs that naturally build on those self-references.
Discuss point of learning goals.
Note: this learning goal is not yet looking for formal inductive proofs. Instead, we’re just exploring how we work with things that are defined in terms of themselves.
61Slide62
Next Lecture Learning Goals: Pre-Class
By the start of class, you should be able to:Given a theorem to prove and the insight into how to break the problem down in terms of smaller problems
, write out the skeleton of an inductive proof including: the base case(s) that need to be proven, the induction hypothesis, and the inductive step that needs to be proven.
Discuss point of learning goals.
So, take what we did in this lecture and use the textbook readings to formalize your understanding.
We will have much more practice on inductive proofs.
62Slide63
Next Lecture Prerequisites
See the Mathematical Induction Textbook Sections at the bottom of the “Textbook and References” page.
63Slide64
Extra Slides
64Slide65
One Extra Step We
Sometimes Do
Really, we are done. But just to be thorough, we’ll add:
Termination:
the number of rounds
n is a non-negative integer, and each application of the inductive step reduces it by at least 1. Therefore, it must reach our base case (0) in a finite number of steps.
65Slide66
Binary Tree Data Definition
A binary-tree is one of:An empty tree(make-node binary-tree
binary-tree number)
66
20
9
2
15
5
10
17
7Slide67
Binary Trees’ Height
A binary tree’s “height” is the maximum number of steps it takes to get from the root down to a node with only empty trees as children.
This one’s height is:
0
1
234
67
20
9
2
15
5
10
17
7Slide68
Binary Trees’ Height
Problem 1: What’s the maximum number of nodes in a binary tree of height n?
68
20
9
2
15
5
10
17
7Slide69
Binary Trees’ Height
Problem 2: Prove your result.
69
20
9
2
15
5
10
17
7Slide70
Binary Trees, Take Two
Problem 1: Give an algorithm to determine the height of a binary tree.
Problem 2
: Prove that your algorithm is correct.
70
20
9
2
15
5
10
17
7Slide71
Worked: Algorithm for Height
Height(t): Is t an empty tree?
Yes: evaluate to -1
No: recursively find the height of
each subtree
hl and hr and evaluate to max(hl, hr) + 1.
71
20
9
2
15
5
10
17
7
How big are the subtrees?
We need to assume the algorithm works on trees of “those sizes”.Slide72
Worked: Proof of Algorithm’s Correctness
Proof: We proceed by induction.
Base case
: On an empty tree, our algorithm yields -1. We simply make this correct
by definition
. (Or, you can imagine we go “up one level” to get to a node.)Induction Hypothesis: For an arbitrary (finite-sized) non-empty tree t, assume our algorithm works correctly on all trees smaller than t.Inductive step: t is not an empty tree; so, our algorithm finds the height of each subtree hl and hr. These heights are correct by assumption, since the subtrees must be smaller than t (lacking at least t itself!). To reach a node with only leaves as children, we go into either the left or right subtree. The length of the path through the left subtree is therefore hl+1, with the +1 for the step down into that subtree. Similarly, the right path is hr
+1. The height is the larger of these possible paths; so, it’s max(hl+1, hr+1) = max(hl, hr) + 1, which is what our algorithm evaluates to. Termination: We call this only on finite trees, and we reduce the size of the tree by at least one node at each inductive step. Thus, we eventually reach
an empty tree.
QED
72
20
9
2
15
5
10
17
7Slide73
Worked: Proof of Algorithm’s Correctness
Proof: We proceed by induction.
Base case
: On an empty tree, our algorithm yields -1. We simply make this correct
by definition
. (Or, you can imagine we go “up one level” to get to a node.)Induction Hypothesis: For an arbitrary (finite-sized) non-empty tree t, assume our algorithm works correctly on all trees smaller than t.Inductive step: t is not an empty tree; so, our algorithm finds the height of each subtree hl and hr. These heights are correct by assumption, since the subtrees must be smaller than t (lacking at least t itself!).
73
20
9
2
15
5
10
17
7Slide74
Worked: Proof of Algorithm’s Correctness
To reach a node with only leaves as children, we go into either the left or right subtree. The length of the path through the left subtree is therefore
h
l
+1, with the +1 for the step down into that subtree. Similarly, the right path is
hr +1. The height is the larger of these possible paths; so, it’s max(hl+1, hr+1) = max(hl, hr) + 1, which is what our algorithm evaluates to. Termination: We call this only on finite trees, and we reduce the size of the tree by at least one node at each inductive step. Thus, we eventually reach an empty tree.QED
74
20
9
2
15
5
10
17
7